-- Logs begin at Thu 2022-09-01 16:52:37 UTC, end at Thu 2022-09-01 17:57:57 UTC. -- Sep 01 17:21:56.677722 dv-cinder-839768-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 17:21:56.689425 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 17:21:56 2022] *** Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: nodename: dv-cinder-839768-6-iscsi Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: machine: x86_64 Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: clock source: unix Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: pcre jit disabled Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: detected number of CPU cores: 4 Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: current working directory: / Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: detected binary path: /usr/bin/uwsgi-core Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: your processes number limit is 47680 Sep 01 17:21:56.691550 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: your memory page size is 4096 bytes Sep 01 17:21:56.692195 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: detected max file descriptor number: 2048 Sep 01 17:21:56.692195 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: lock engine: pthread robust mutexes Sep 01 17:21:56.692195 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: thunder lock: enabled Sep 01 17:21:56.692195 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 17:21:56.692195 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 17:21:56.734192 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: Python main interpreter initialized at 0x5650d36a99c0 Sep 01 17:21:56.734192 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: python threads support enabled Sep 01 17:21:56.734192 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: your server socket listen backlog is limited to 100 connections Sep 01 17:21:56.734192 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: your mercy for graceful operations on workers is 90 seconds Sep 01 17:21:56.734641 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 17:21:56.734641 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: *** Operational MODE: preforking *** Sep 01 17:21:56.734913 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: *** uWSGI is running in multiple interpreter mode *** Sep 01 17:21:56.734913 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: spawned uWSGI master process (pid: 81519) Sep 01 17:21:56.735133 dv-cinder-839768-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 17:21:56.737961 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: spawned uWSGI worker 1 (pid: 81532, cores: 1) Sep 01 17:21:56.737961 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: spawned uWSGI worker 2 (pid: 81533, cores: 1) Sep 01 17:21:56.737961 dv-cinder-839768-6-iscsi devstack@keystone.service[81519]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 17:22:01.106888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:22:01.106888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: from cryptography.utils import int_from_bytes Sep 01 17:22:01.107682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:22:01.107682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: from cryptography.utils import int_from_bytes Sep 01 17:22:01.108141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:22:01.108141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: from cryptography.utils import int_from_bytes Sep 01 17:22:01.108279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:22:01.108279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: from cryptography.utils import int_from_bytes Sep 01 17:22:01.158764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 17:22:01.158969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 17:22:01.159131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] command line args: [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 17:22:01.159382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 17:22:01.159557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ================================================================================ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 17:22:01.159897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] admin_token = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.159996 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 17:22:01.160121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config_dir = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.160346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 17:22:01.160483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.160714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] command line args: [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 17:22:01.160793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config_source = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.160865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 17:22:01.160929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] debug = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.160997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ================================================================================ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 17:22:01.161058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.161235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] admin_token = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.161299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.161356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config_dir = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.161613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.161823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.161887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.161950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config_source = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.161950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] debug = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.162062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.162130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.162219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.162279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.162776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_config_append = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.162866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.162939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.162998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.163055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_dir = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.163116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.163179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.163312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.163552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.163552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.163798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_config_append = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.163923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_dir = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.164926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.165980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.166061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.166183 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.166259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.166333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.166466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.166552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.166746 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.100/identity {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.166883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.166978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] publish_errors = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.167052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.167168 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.167308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.167398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.167516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.167765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.167862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.100/identity {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.167980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] publish_errors = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.168069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.168216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] standard_threads = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.168301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.168447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.168540 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.168611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.168671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.168851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.168939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.169023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_journal = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.169091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.169240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_json = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.169324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] standard_threads = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.169384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_stderr = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.169521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_syslog = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.169640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.169776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.169855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.170039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.170163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.170248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_journal = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.170309 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.170448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_json = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.170554 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.170692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_stderr = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.170776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.170954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_syslog = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.171184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.171184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.171352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:22:01.171464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.171671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.171800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.171924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.172003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.172191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.172280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.172460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.172539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.172834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.173033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.173164 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.173288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.173709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.173709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.174341 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.174437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.174664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.174787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.174998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.175105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.175184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.175263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.176294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.176407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.176583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.176655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.176713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.176777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.176919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.176988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.176988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.177963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.178021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.178077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.178139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.178197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.178261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.178320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.178377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180758 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.180938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.181962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.182973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.183138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.183244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.183368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.183478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.183617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.183722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.183853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.183963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.184088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.184221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.184336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.184434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.184567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.184670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.184800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.184881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.185033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.185116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.185269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.185350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.185504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.185649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.185767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.185859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.186007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.186093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.186306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.186399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.186535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.186629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.186759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.186899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.187972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.188126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.188284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.188411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.188487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.188601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.188685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.188817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.188897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.189052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.189131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.189288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.189375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.189525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.189634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.189787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.189865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.190038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.190172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.190281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.190360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.190522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.190605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.190761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.190850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.190998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.191077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.191238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.191321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.191472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.191557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.191713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.191791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.191948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.192027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.192176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.192334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.192416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.192488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.192622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.192702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.192856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.192943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.193086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.193174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.193903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.193995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.194144 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.194394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.194485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.194632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.194711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.194884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.195027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.195152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.195290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.195419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.195546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.195673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.195811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.external = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.195936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.196071 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.196204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.196329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.196452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.196578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.196710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.196852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.password = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.196979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.197109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.token = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.197261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.197417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.197544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.197636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.197709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.197793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.197867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.197938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.198014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.198091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.198163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.198296 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.198378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.198548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.198634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.198788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.external = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.198868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.199027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.199108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.199295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.199373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.199540 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.199618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.199780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.password = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.199865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.200013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.200090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.token = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.200257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.200334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.200501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.200592 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.200736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.200814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.200980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.201064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.201233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.201313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.201485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.201641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.201746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.201825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.201988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.202069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.202242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.202321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.202478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.202555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.202716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.202797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.202962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.203040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.203209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.203286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.203438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.203517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.203686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.203766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.203926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.204004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.204168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.204251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.204407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.204491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.204662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.204741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.204900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.204987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.205146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.205225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.205383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.205461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.205654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.205737 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.205903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.205983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.206140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.206231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.206376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.206462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.206624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.206704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.206857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.207011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.207088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.207159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.207330 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.207409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.207582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.207668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.207818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.207933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.208050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.208144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.208281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.208369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.208526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.208606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.208772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.208852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.209145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.209812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.209959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.210031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.210209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.210290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.210485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.210576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.210724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.210806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.210961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.211044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.211192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.211283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.211428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.211510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.211657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.211739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.211892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.211982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.212134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.212216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.212377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.212455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.212622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.212712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.212852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.212933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.213097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.213179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.213342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.213432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.213630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.213717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.213840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.213916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.214079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.214174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.214294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.214371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.214532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.214614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.214760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.214848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.214996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.215078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.215233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.215313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.215473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.215562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.215707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.215787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.216488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.216592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.216592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.216829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.216909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.217961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.218888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219154 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.219985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.220066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.220211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.220293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.220448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.220529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.220699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.220779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.220940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.221021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.221180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.221264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.221422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.221537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.221676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.221813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.221906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.222061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.222164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.222314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.222393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.222641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.222731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.222879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.222957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.223126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.223207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.223363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.223452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.223598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.223678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.223833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.223913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.224068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.224158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.224308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.224391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.224593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.224675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.224911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.225045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.225178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.225315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.225502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.225617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.225806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.226045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.226138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.226349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.226430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.226613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.226705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.226919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.227003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.227166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.227313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.227413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.227651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.227744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.227895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.227977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.228147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.228229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.228402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.228493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.228661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.228750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.228903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.228985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.229170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.229322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.229448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.229663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.229746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.229919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.230044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.230182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.230308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.230435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.230564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.230692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.230819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.230955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.231077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.231211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.231336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.231460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.231588 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.231797 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.231921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.232048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.232174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.232298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.232421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.232553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.232677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.232801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.232934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.233060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.233192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.233318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.233439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.233565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.233708 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.233784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.233913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.233992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.234963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235086 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.235942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.236057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.236133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.236257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.236388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.236468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.236663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.236854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.236960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.237048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.237169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.237246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.237349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.237540 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.237639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.237804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.237885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.238055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.238136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.238294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.238385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.238594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.238683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.238801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.238877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.238992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.239077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.239209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.239288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.239456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.239538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.239694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.239783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.239936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.240021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.240197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.240281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.240434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.240532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.240675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.240761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.240923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.241027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.241172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.241289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.241407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.241533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.241699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.241835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.241961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.242098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.242223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.242359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.242485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.242613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.242737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.242879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.243011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.243142 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.243265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.243390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.243514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.243640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.243773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.243906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.244031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.244165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.244290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.244414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.244549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.244672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.244795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.244923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.245047 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.245174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.245305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.245429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.245561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.245756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.245893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.246016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.246154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.246277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.246411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.246544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.246669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.246793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.246915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.247001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.247129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.247251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.247376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.247376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.247620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.247743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.247878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.248001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.248125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.248246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 17:22:01.248372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.248372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.248372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.248372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.248826 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.248826 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.249060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.249264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.249517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.249784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.250032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.250264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.250507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.250744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:22:01.250915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 17:22:01.278790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.279069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.279236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.281788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.282055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.282339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.288694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.291575 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.317991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.319026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.320488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.320720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.377339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.377625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.430024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.430430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.430953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.431176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.598115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.598573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.598815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.599033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.599246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.599453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.600281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.600524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.600727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.600978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.601188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.601188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.624992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.625272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.625418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.625557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.625731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.625836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.625917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.626037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.626118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.626240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.626318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.626448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.626528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.626640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.638649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.638919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.652809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.652994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.653067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.653224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.653311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.653457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.653553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.653705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.653784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.653920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.653997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.654140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.654218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.654359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:22:01.671757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.672006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:22:01.672219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.672374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.672514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:22:01.672661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:22:01.672793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.672994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.673170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.673268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:22:01.673344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.673500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.673632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.673793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.674059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.674323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.683810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.684095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.684373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.684505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.684657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.684800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.685096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.685366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.692116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.692423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.695681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.695963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.696093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.696247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.696383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.696530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.696669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.696813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.696955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.697075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.697238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.697344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.697506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.697688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.697909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.697992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.698190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.698285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.698451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.698575 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.698817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.699094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.722347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.722521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.723842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.723950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.724136 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.724231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.729552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.729768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.729858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.729985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.730135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.730220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.736648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.736767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.736935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.737022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.741055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.741159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.741345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.741430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.741703 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.741808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.741989 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.742091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.742273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.742367 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.742576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.742853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.743126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.744445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.744732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.745016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.761278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.761469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.761607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.761812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.765458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.765700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.768604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.768836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.768929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.769157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.774204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.774461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.777352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.777783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.780485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.780781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.780904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.781068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.781211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.781347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.781507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.781666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.781809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.782039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.782156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.782355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.798096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.798389 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.798561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.798659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.798860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.798948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.799135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.799402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.805753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.806671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.807419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.808215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.808968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.809244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.809844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.810128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.811909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.812193 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.812487 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.812767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.813052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.813204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.813495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.813816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.814117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.825034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.825325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.825627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.825908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.826045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.826190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.826349 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.826482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.826644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.826786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.826923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.827199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.827476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.827784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.843157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.844035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.846243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.847005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.849143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.849967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.851117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.851650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.852291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.852938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.852938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.853107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.853297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.853614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.860045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.860709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.861749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.862227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.864654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.864961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.865107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.865315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.865473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.865624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.865789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.866064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.873731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.874009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.874096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.874299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.874391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.874579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.881828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.882031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:22:01.882136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.882311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.882395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.882597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.882721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.882877 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.882960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.883149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.883234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.883420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.883566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.883699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.883777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.883960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.884116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.884237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.884405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.884488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.884686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.884767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.884957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.885046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.885242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.885350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.885515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.885628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:22:01.920958 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5650d36a99c0 pid: 81532 (default app) Sep 01 17:22:01.921146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5650d36a99c0 pid: 81533 (default app) Sep 01 17:22:01.924550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb48ef-eb76-4a37-9f81-ddcf4574b4ad None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:01.924846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb48ef-eb76-4a37-9f81-ddcf4574b4ad None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:01.925120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb48ef-eb76-4a37-9f81-ddcf4574b4ad None None] PATH_INFO: `/v3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:01.929723 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1/1] 10.222.0.100 () {50 vars in 796 bytes} [Thu Sep 1 17:22:01 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:08.700860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d8276-60eb-4c45-a216-0e63579fcaa5 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:08.701419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d8276-60eb-4c45-a216-0e63579fcaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:08.701419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d8276-60eb-4c45-a216-0e63579fcaa5 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:08.705831 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1/2] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:08 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:08.711760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:08.712405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:08.712405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:08.783704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: INFO dbcounter [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] Registered counter for database keystone Sep 01 17:22:08.804501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG oslo_db.sqlalchemy.engines [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 17:22:08.810832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writer thread running {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 17:22:08.997545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG passlib.handlers.bcrypt [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81533) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 17:22:08.997897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG passlib.handlers.bcrypt [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81533) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 17:22:09.057477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:09.059167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:09.200992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 2/3] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:08 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 17:22:09.688324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d8276-60eb-4c45-a216-0e63579fcaa5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:09.758458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: INFO dbcounter [None req-bc8d8276-60eb-4c45-a216-0e63579fcaa5 None None] Registered counter for database keystone Sep 01 17:22:09.780603 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG oslo_db.sqlalchemy.engines [None req-bc8d8276-60eb-4c45-a216-0e63579fcaa5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81532) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 17:22:09.787248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writer thread running {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 17:22:09.920568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d8276-60eb-4c45-a216-0e63579fcaa5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:09.924923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2faf5-6720-4ff6-93a1-f418d0ed9ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:09.925558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2faf5-6720-4ff6-93a1-f418d0ed9ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:09.925863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2faf5-6720-4ff6-93a1-f418d0ed9ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:09.926135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2faf5-6720-4ff6-93a1-f418d0ed9ecd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:09.935462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2faf5-6720-4ff6-93a1-f418d0ed9ecd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:09.989157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WARNING keystone.common.rbac_enforcer.enforcer [None req-54f2faf5-6720-4ff6-93a1-f418d0ed9ecd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 17:22:09.989499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2faf5-6720-4ff6-93a1-f418d0ed9ecd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:10.000522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 2/4] 10.222.0.100 () {58 vars in 1329 bytes} [Thu Sep 1 17:22:09 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:22:10.006623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:10.021036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b958b-224b-499a-b416-d5d7619dad5d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:10.025774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6affebf-e713-48f9-847f-3f2456dc8459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:10.026880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6affebf-e713-48f9-847f-3f2456dc8459 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:10.027159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6affebf-e713-48f9-847f-3f2456dc8459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:10.027430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6affebf-e713-48f9-847f-3f2456dc8459 admin admin] PATH_INFO: `/v3/projects/392e00333a6b41639a27c8ea706c3db5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:10.028973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6affebf-e713-48f9-847f-3f2456dc8459 admin admin] RBAC: Authorizing `identity:get_project(project_id=392e00333a6b41639a27c8ea706c3db5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:10.083286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a6affebf-e713-48f9-847f-3f2456dc8459 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 17:22:10.083632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6affebf-e713-48f9-847f-3f2456dc8459 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:10.088593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 3/5] 10.222.0.100 () {56 vars in 1212 bytes} [Thu Sep 1 17:22:10 2022] GET /identity/v3/projects/392e00333a6b41639a27c8ea706c3db5 => generated 345 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:12.015913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73395e27-b5fe-41ab-a030-bbf253d17fd1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:12.016426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73395e27-b5fe-41ab-a030-bbf253d17fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:12.016426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73395e27-b5fe-41ab-a030-bbf253d17fd1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:12.020511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 3/6] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:12 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:12.027264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a75cf3-7a74-4693-a38f-06d093c985ad None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:12.027726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a75cf3-7a74-4693-a38f-06d093c985ad None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:12.027826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a75cf3-7a74-4693-a38f-06d093c985ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:12.049129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c0a75cf3-7a74-4693-a38f-06d093c985ad None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:12.050725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c0a75cf3-7a74-4693-a38f-06d093c985ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:12.058903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 4/7] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:12 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 17:22:12.334195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73395e27-b5fe-41ab-a030-bbf253d17fd1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:12.346805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73395e27-b5fe-41ab-a030-bbf253d17fd1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:12.351151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc94cc-d983-4758-93cf-432f99218af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:12.351747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc94cc-d983-4758-93cf-432f99218af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:12.352026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc94cc-d983-4758-93cf-432f99218af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:12.352297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc94cc-d983-4758-93cf-432f99218af4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:12.360451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc94cc-d983-4758-93cf-432f99218af4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:12.362243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc94cc-d983-4758-93cf-432f99218af4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:12.371945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 4/8] 10.222.0.100 () {58 vars in 1329 bytes} [Thu Sep 1 17:22:12 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 17:22:12.377737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a75cf3-7a74-4693-a38f-06d093c985ad None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:12.387870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a75cf3-7a74-4693-a38f-06d093c985ad None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:12.392260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0144eb-62c5-4cde-959e-228a8d3b31a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:12.392881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0144eb-62c5-4cde-959e-228a8d3b31a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:12.393158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0144eb-62c5-4cde-959e-228a8d3b31a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:12.393426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0144eb-62c5-4cde-959e-228a8d3b31a1 admin admin] PATH_INFO: `/v3/users/d51a342066d64ae3b06abd4744cb8e8b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:12.394757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0144eb-62c5-4cde-959e-228a8d3b31a1 admin admin] RBAC: Authorizing `identity:get_user(user_id=d51a342066d64ae3b06abd4744cb8e8b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:12.396651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0144eb-62c5-4cde-959e-228a8d3b31a1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:12.399483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 5/9] 10.222.0.100 () {56 vars in 1203 bytes} [Thu Sep 1 17:22:12 2022] GET /identity/v3/users/d51a342066d64ae3b06abd4744cb8e8b => generated 248 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:14.405895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e53603-e9f7-4df6-a335-77f561eabd55 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:14.406471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e53603-e9f7-4df6-a335-77f561eabd55 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:14.406471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e53603-e9f7-4df6-a335-77f561eabd55 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:14.407762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 5/10] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:14.413527 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce03013-1fa0-4f1a-a02c-1c806a8854cf None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:14.414009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce03013-1fa0-4f1a-a02c-1c806a8854cf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:14.414117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce03013-1fa0-4f1a-a02c-1c806a8854cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:14.434391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3ce03013-1fa0-4f1a-a02c-1c806a8854cf None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:14.434661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccc7b3-217d-417b-a5aa-7c2bc8dac73b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:14.434954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccc7b3-217d-417b-a5aa-7c2bc8dac73b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:14.435237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccc7b3-217d-417b-a5aa-7c2bc8dac73b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:14.435604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3ce03013-1fa0-4f1a-a02c-1c806a8854cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:14.436632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 6/11] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:22:14.442505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe61d7-5921-4d4c-9786-3dcc0d923f4f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:14.442797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe61d7-5921-4d4c-9786-3dcc0d923f4f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:14.443078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe61d7-5921-4d4c-9786-3dcc0d923f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:14.487284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG passlib.handlers.bcrypt [None req-3bfe61d7-5921-4d4c-9786-3dcc0d923f4f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81532) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 17:22:14.487681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG passlib.handlers.bcrypt [None req-3bfe61d7-5921-4d4c-9786-3dcc0d923f4f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81532) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 17:22:14.521443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 6/12] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:14.538522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3bfe61d7-5921-4d4c-9786-3dcc0d923f4f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:14.540146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3bfe61d7-5921-4d4c-9786-3dcc0d923f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:14.670368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 7/13] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:14.818277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce03013-1fa0-4f1a-a02c-1c806a8854cf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:14.847288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:14.942464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce03013-1fa0-4f1a-a02c-1c806a8854cf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:14.956022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe61d7-5921-4d4c-9786-3dcc0d923f4f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:14.981307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9b87-772a-4bc9-8c73-5e20bfe716a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:14.981983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9b87-772a-4bc9-8c73-5e20bfe716a5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:14.982264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9b87-772a-4bc9-8c73-5e20bfe716a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:14.982541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9b87-772a-4bc9-8c73-5e20bfe716a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:14.994376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c9b87-772a-4bc9-8c73-5e20bfe716a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a36c9b87-772a-4bc9-8c73-5e20bfe716a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:15.003380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:15.004349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:15.005163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 7/14] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:22:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:15.010888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9b87-772a-4bc9-8c73-5e20bfe716a5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:15.073022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe61d7-5921-4d4c-9786-3dcc0d923f4f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:15.075553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:15.096092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:15.114186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303fcf4-1733-4b57-818a-41ef86228908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:15.115034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303fcf4-1733-4b57-818a-41ef86228908 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:15.115335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303fcf4-1733-4b57-818a-41ef86228908 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:15.115610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303fcf4-1733-4b57-818a-41ef86228908 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:15.118626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9b87-772a-4bc9-8c73-5e20bfe716a5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:15.160526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a4ead-7f49-441a-8923-f12e096de63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:15.161031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a4ead-7f49-441a-8923-f12e096de63e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:15.161320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a4ead-7f49-441a-8923-f12e096de63e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:15.161613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a4ead-7f49-441a-8923-f12e096de63e None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:15.162911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a4ead-7f49-441a-8923-f12e096de63e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:15.163803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a4ead-7f49-441a-8923-f12e096de63e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:15.164705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a4ead-7f49-441a-8923-f12e096de63e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:15.177325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 8/15] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:15 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:15.183035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a4ead-7f49-441a-8923-f12e096de63e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:15.189188 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2303fcf4-1733-4b57-818a-41ef86228908 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:15.191080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2303fcf4-1733-4b57-818a-41ef86228908 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:15.287695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a4ead-7f49-441a-8923-f12e096de63e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:15.297519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 8/16] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:22:15.303174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303fcf4-1733-4b57-818a-41ef86228908 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:15.306187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:15.327882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147af35-d2b7-48d5-bdcc-a0b4053b4052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:15.328611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147af35-d2b7-48d5-bdcc-a0b4053b4052 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:15.328893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147af35-d2b7-48d5-bdcc-a0b4053b4052 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:15.329166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147af35-d2b7-48d5-bdcc-a0b4053b4052 None admin] PATH_INFO: `/v3/users/d51a342066d64ae3b06abd4744cb8e8b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:15.330383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147af35-d2b7-48d5-bdcc-a0b4053b4052 None admin] RBAC: Authorizing `identity:get_user(user_id=d51a342066d64ae3b06abd4744cb8e8b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:15.331848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147af35-d2b7-48d5-bdcc-a0b4053b4052 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:15.333981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 9/17] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:15 2022] GET /identity/v3/users/d51a342066d64ae3b06abd4744cb8e8b => generated 248 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:15.340250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147af35-d2b7-48d5-bdcc-a0b4053b4052 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:15.356926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:15.419480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303fcf4-1733-4b57-818a-41ef86228908 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:15.460646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147af35-d2b7-48d5-bdcc-a0b4053b4052 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:15.461208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e4b0d-12b6-4e5d-ad89-6c7b0d49c7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:15.461866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e4b0d-12b6-4e5d-ad89-6c7b0d49c7fa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:15.462151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e4b0d-12b6-4e5d-ad89-6c7b0d49c7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:15.462420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e4b0d-12b6-4e5d-ad89-6c7b0d49c7fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:15.481082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e4b0d-12b6-4e5d-ad89-6c7b0d49c7fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:15.482583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e4b0d-12b6-4e5d-ad89-6c7b0d49c7fa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:15.494139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcf5dd-2662-45c6-ad16-f29d1ab3aa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:15.494782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcf5dd-2662-45c6-ad16-f29d1ab3aa24 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:15.495061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcf5dd-2662-45c6-ad16-f29d1ab3aa24 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:15.495332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcf5dd-2662-45c6-ad16-f29d1ab3aa24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:15.496606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcf5dd-2662-45c6-ad16-f29d1ab3aa24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:15.498103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcf5dd-2662-45c6-ad16-f29d1ab3aa24 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:15.499878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 10/18] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:15 2022] GET /identity/v3/domains/default => generated 200 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6f9e4b0d-12b6-4e5d-ad89-6c7b0d49c7fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:22:15.501926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:15.502863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:22:15.503672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:22:15.503672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:22:15.503672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:15.503672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 9/19] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:22:15.505618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcf5dd-2662-45c6-ad16-f29d1ab3aa24 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:15.510813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e4b0d-12b6-4e5d-ad89-6c7b0d49c7fa None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:15.573144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:15.594318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:15.616865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcf5dd-2662-45c6-ad16-f29d1ab3aa24 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:15.624870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e4b0d-12b6-4e5d-ad89-6c7b0d49c7fa None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:15.655881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd9f74-4081-4db0-9431-b680f9a4f3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:15.656411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9f74-4081-4db0-9431-b680f9a4f3be None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:15.656687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9f74-4081-4db0-9431-b680f9a4f3be None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:15.656956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9f74-4081-4db0-9431-b680f9a4f3be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:15.657718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd9f74-4081-4db0-9431-b680f9a4f3be None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.domain.id=default, user.id=d51a342066d64ae3b06abd4744cb8e8b) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:15.658025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd9f74-4081-4db0-9431-b680f9a4f3be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:15.659478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd9f74-4081-4db0-9431-b680f9a4f3be None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:15.663148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b761d78-072a-4fee-817c-313f14e5ecf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:15.663768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b761d78-072a-4fee-817c-313f14e5ecf5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:15.664051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b761d78-072a-4fee-817c-313f14e5ecf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:15.664324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b761d78-072a-4fee-817c-313f14e5ecf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:15.664981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b761d78-072a-4fee-817c-313f14e5ecf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:15.665285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b761d78-072a-4fee-817c-313f14e5ecf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:15.666719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b761d78-072a-4fee-817c-313f14e5ecf5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:15.667373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 11/20] 10.222.0.100 () {56 vars in 1327 bytes} [Thu Sep 1 17:22:15 2022] GET /identity/v3/role_assignments?user.id=d51a342066d64ae3b06abd4744cb8e8b&scope.domain.id=default&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 234 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:15.687763 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 10/21] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:22:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:18.144275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df7c24-b346-446a-88bd-5b46fafc1593 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:18.144804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df7c24-b346-446a-88bd-5b46fafc1593 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:18.145083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df7c24-b346-446a-88bd-5b46fafc1593 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:18.147715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 12/22] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:18 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:18.153383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f57eb2-4c14-4f33-9ec5-1e34575ba820 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:18.153874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f57eb2-4c14-4f33-9ec5-1e34575ba820 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:18.153946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f57eb2-4c14-4f33-9ec5-1e34575ba820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:18.182552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a3f57eb2-4c14-4f33-9ec5-1e34575ba820 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:18.183872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a3f57eb2-4c14-4f33-9ec5-1e34575ba820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:18.267944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 11/23] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:18.504123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34aa1d-95f5-45c5-8249-eabf2f622ae2 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:18.504489 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34aa1d-95f5-45c5-8249-eabf2f622ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:18.504617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34aa1d-95f5-45c5-8249-eabf2f622ae2 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:18.505963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 13/24] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:18.511698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815b3a4-6adf-4f8b-9327-b1bffcadbc8b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:18.512001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815b3a4-6adf-4f8b-9327-b1bffcadbc8b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:18.512273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815b3a4-6adf-4f8b-9327-b1bffcadbc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:18.536288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4815b3a4-6adf-4f8b-9327-b1bffcadbc8b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:18.538190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4815b3a4-6adf-4f8b-9327-b1bffcadbc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:18.590098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:18.626593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 12/25] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:18.783976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2abbd4-8064-41a6-854f-c22c0163b4e5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:18.784159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2abbd4-8064-41a6-854f-c22c0163b4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:18.784436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2abbd4-8064-41a6-854f-c22c0163b4e5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:18.785670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815b3a4-6adf-4f8b-9327-b1bffcadbc8b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:18.785852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 14/26] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:18.798419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a895c74-073b-4c77-a8cd-7382b25900c9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:18.798731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a895c74-073b-4c77-a8cd-7382b25900c9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:18.798984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a895c74-073b-4c77-a8cd-7382b25900c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:18.831289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1a895c74-073b-4c77-a8cd-7382b25900c9 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:18.832694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1a895c74-073b-4c77-a8cd-7382b25900c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:18.976655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 15/27] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:18.979046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815b3a4-6adf-4f8b-9327-b1bffcadbc8b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:18.982264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da0117-6075-40e9-a52e-48af7a3f03c7 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:18.983057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da0117-6075-40e9-a52e-48af7a3f03c7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:18.984436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da0117-6075-40e9-a52e-48af7a3f03c7 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:18.986086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 16/28] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:18 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:18.993087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e0795-de1d-49a5-ada9-8addae4b2751 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:18.993724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e0795-de1d-49a5-ada9-8addae4b2751 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:18.994203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e0795-de1d-49a5-ada9-8addae4b2751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:19.020040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ab9e0795-de1d-49a5-ada9-8addae4b2751 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:19.021251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ab9e0795-de1d-49a5-ada9-8addae4b2751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:19.042683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587deba-3bf3-48fe-84ef-c39c95575223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:19.043685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587deba-3bf3-48fe-84ef-c39c95575223 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:19.044186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587deba-3bf3-48fe-84ef-c39c95575223 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:19.044705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587deba-3bf3-48fe-84ef-c39c95575223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:19.056909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587deba-3bf3-48fe-84ef-c39c95575223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:19.058787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587deba-3bf3-48fe-84ef-c39c95575223 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8587deba-3bf3-48fe-84ef-c39c95575223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:22:19.075699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:19.076639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:22:19.077459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:22:19.077459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:22:19.077459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:19.079714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 13/29] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:19.080749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587deba-3bf3-48fe-84ef-c39c95575223 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:19.124352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:19.140145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 17/30] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:19.140898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e0795-de1d-49a5-ada9-8addae4b2751 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:19.151760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:19.295936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587deba-3bf3-48fe-84ef-c39c95575223 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:19.347266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a0087-648c-4e3d-89ff-ce4609824f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:19.347923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a0087-648c-4e3d-89ff-ce4609824f17 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:19.348210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a0087-648c-4e3d-89ff-ce4609824f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:19.348487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a0087-648c-4e3d-89ff-ce4609824f17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:19.371430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a0087-648c-4e3d-89ff-ce4609824f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:19.377681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e0795-de1d-49a5-ada9-8addae4b2751 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-950a0087-648c-4e3d-89ff-ce4609824f17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:19.387220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:19.388139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:19.388994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 14/31] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:22:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:19.394752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a0087-648c-4e3d-89ff-ce4609824f17 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:19.465137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfeb98-60c6-48dc-967d-90610b934d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:19.465807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfeb98-60c6-48dc-967d-90610b934d42 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:19.466093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfeb98-60c6-48dc-967d-90610b934d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:19.466371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfeb98-60c6-48dc-967d-90610b934d42 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:19.467037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfeb98-60c6-48dc-967d-90610b934d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:19.467344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfeb98-60c6-48dc-967d-90610b934d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:19.468754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfeb98-60c6-48dc-967d-90610b934d42 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:19.501078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 18/32] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:22:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:19.508937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:19.514733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfeb98-60c6-48dc-967d-90610b934d42 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:19.522981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a0087-648c-4e3d-89ff-ce4609824f17 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:19.560015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aec472-94a5-43ac-bd4d-03c7260cf677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:19.560650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aec472-94a5-43ac-bd4d-03c7260cf677 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:19.560829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aec472-94a5-43ac-bd4d-03c7260cf677 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:19.561086 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aec472-94a5-43ac-bd4d-03c7260cf677 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:19.561799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aec472-94a5-43ac-bd4d-03c7260cf677 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:19.562201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aec472-94a5-43ac-bd4d-03c7260cf677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:19.563504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aec472-94a5-43ac-bd4d-03c7260cf677 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:19.582099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 15/33] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:19 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:19.583027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aec472-94a5-43ac-bd4d-03c7260cf677 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:19.605378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:19.666096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfeb98-60c6-48dc-967d-90610b934d42 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:19.694780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aec472-94a5-43ac-bd4d-03c7260cf677 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:19.723429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de219ada-f593-4c2b-a2e6-f95dcf751a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:19.724460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de219ada-f593-4c2b-a2e6-f95dcf751a78 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:19.725495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de219ada-f593-4c2b-a2e6-f95dcf751a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:19.726008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de219ada-f593-4c2b-a2e6-f95dcf751a78 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:19.727109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de219ada-f593-4c2b-a2e6-f95dcf751a78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:19.729707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de219ada-f593-4c2b-a2e6-f95dcf751a78 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:19.734681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b2ed-bb4c-4433-a4fa-150e600ec624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:19.735557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b2ed-bb4c-4433-a4fa-150e600ec624 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:19.736055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b2ed-bb4c-4433-a4fa-150e600ec624 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:19.736510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b2ed-bb4c-4433-a4fa-150e600ec624 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:19.737412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3b2ed-bb4c-4433-a4fa-150e600ec624 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:19.739034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3b2ed-bb4c-4433-a4fa-150e600ec624 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:19.863922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 16/34] 10.222.0.100 () {60 vars in 1136 bytes} [Thu Sep 1 17:22:19 2022] POST /identity/v3/roles => generated 220 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:22:19.864829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b2ed-bb4c-4433-a4fa-150e600ec624 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:19.868228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 19/35] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:22:19 2022] POST /identity/v3/projects => generated 302 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:22:19.873641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de219ada-f593-4c2b-a2e6-f95dcf751a78 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:19.890047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:20.014440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b2ed-bb4c-4433-a4fa-150e600ec624 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:20.052478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de219ada-f593-4c2b-a2e6-f95dcf751a78 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:20.070546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0b7a9-537a-4a8f-91f0-02322b9717f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:20.071420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0b7a9-537a-4a8f-91f0-02322b9717f4 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.071853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0b7a9-537a-4a8f-91f0-02322b9717f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.072293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0b7a9-537a-4a8f-91f0-02322b9717f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.073213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0b7a9-537a-4a8f-91f0-02322b9717f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:20.074971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0b7a9-537a-4a8f-91f0-02322b9717f4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:20.077842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:20.093739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 17/36] 10.222.0.100 () {60 vars in 1136 bytes} [Thu Sep 1 17:22:19 2022] POST /identity/v3/roles => generated 226 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:22:20.095160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac62c55-15db-4bb5-9796-13211fccdac8 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.095502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac62c55-15db-4bb5-9796-13211fccdac8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.095725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac62c55-15db-4bb5-9796-13211fccdac8 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.096999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 18/37] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:20.098192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821f00d-67d2-430f-a065-c995666fc63c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.098481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821f00d-67d2-430f-a065-c995666fc63c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.098757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821f00d-67d2-430f-a065-c995666fc63c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.103911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 19/38] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:20.107065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cd55d-73ec-4e5d-9002-c1256c936b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:20.108022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cd55d-73ec-4e5d-9002-c1256c936b92 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.108315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cd55d-73ec-4e5d-9002-c1256c936b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.108597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cd55d-73ec-4e5d-9002-c1256c936b92 None admin] PATH_INFO: `/v3/users/d51a342066d64ae3b06abd4744cb8e8b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.111200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cd55d-73ec-4e5d-9002-c1256c936b92 None admin] RBAC: Authorizing `identity:get_user(user_id=d51a342066d64ae3b06abd4744cb8e8b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:20.112394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936313f6-0ed7-4388-a95c-973b04f3e3ae None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.112854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cd55d-73ec-4e5d-9002-c1256c936b92 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:20.113121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936313f6-0ed7-4388-a95c-973b04f3e3ae None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.113612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936313f6-0ed7-4388-a95c-973b04f3e3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.114653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 20/39] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:19 2022] GET /identity/v3/users/d51a342066d64ae3b06abd4744cb8e8b => generated 248 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:20.116570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c9af6-7850-4413-b07d-0dde2f50765e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.116857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c9af6-7850-4413-b07d-0dde2f50765e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.117135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c9af6-7850-4413-b07d-0dde2f50765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.136758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-936313f6-0ed7-4388-a95c-973b04f3e3ae None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:20.138910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-936313f6-0ed7-4388-a95c-973b04f3e3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:20.139799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-590c9af6-7850-4413-b07d-0dde2f50765e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:20.141500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-590c9af6-7850-4413-b07d-0dde2f50765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:20.224455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:20.233650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 20/40] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:20.233650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936313f6-0ed7-4388-a95c-973b04f3e3ae None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:20.239205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 21/41] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:20.358122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936313f6-0ed7-4388-a95c-973b04f3e3ae None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:20.397770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3b623-b1b3-4760-8638-e8d2eb9c1578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:20.398460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3b623-b1b3-4760-8638-e8d2eb9c1578 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.398950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3b623-b1b3-4760-8638-e8d2eb9c1578 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.399209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3b623-b1b3-4760-8638-e8d2eb9c1578 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.400656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3b623-b1b3-4760-8638-e8d2eb9c1578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:20.402207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3b623-b1b3-4760-8638-e8d2eb9c1578 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:20.403988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 21/42] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:20 2022] GET /identity/v3/domains/default => generated 200 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:20.409466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c9af6-7850-4413-b07d-0dde2f50765e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:20.511775 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c9af6-7850-4413-b07d-0dde2f50765e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:20.514303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3b623-b1b3-4760-8638-e8d2eb9c1578 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:20.528430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:20.554686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da5e28-c77a-4c1a-9446-b570f3299807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:20.555362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da5e28-c77a-4c1a-9446-b570f3299807 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.555649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da5e28-c77a-4c1a-9446-b570f3299807 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.555933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da5e28-c77a-4c1a-9446-b570f3299807 None admin] PATH_INFO: `/v3/domains/default/users/d51a342066d64ae3b06abd4744cb8e8b/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.557938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da5e28-c77a-4c1a-9446-b570f3299807 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d51a342066d64ae3b06abd4744cb8e8b, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:20.559404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da5e28-c77a-4c1a-9446-b570f3299807 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:20.574424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 22/43] 10.222.0.100 () {58 vars in 1366 bytes} [Thu Sep 1 17:22:20 2022] PUT /identity/v3/domains/default/users/d51a342066d64ae3b06abd4744cb8e8b/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:22:20.575595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da5e28-c77a-4c1a-9446-b570f3299807 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:20.608188 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:20.653996 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3b623-b1b3-4760-8638-e8d2eb9c1578 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:20.703443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f26983-c84e-4bd9-b07b-70aff4a927e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:20.704379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f26983-c84e-4bd9-b07b-70aff4a927e3 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.704725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f26983-c84e-4bd9-b07b-70aff4a927e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.705006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f26983-c84e-4bd9-b07b-70aff4a927e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.705785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f26983-c84e-4bd9-b07b-70aff4a927e3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:20.707237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f26983-c84e-4bd9-b07b-70aff4a927e3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:20.729721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 22/44] 10.222.0.100 () {60 vars in 1136 bytes} [Thu Sep 1 17:22:20 2022] POST /identity/v3/roles => generated 224 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:22:20.836751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da5e28-c77a-4c1a-9446-b570f3299807 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:20.876623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a762a-ed4a-4b37-a3dc-15be6b52d7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:20.877283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a762a-ed4a-4b37-a3dc-15be6b52d7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:20.877560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a762a-ed4a-4b37-a3dc-15be6b52d7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:20.877884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a762a-ed4a-4b37-a3dc-15be6b52d7c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:20.879067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a762a-ed4a-4b37-a3dc-15be6b52d7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:20.880555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a762a-ed4a-4b37-a3dc-15be6b52d7c0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:20.885608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 23/45] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:20 2022] GET /identity/v3/domains/default => generated 200 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:20.888993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f26983-c84e-4bd9-b07b-70aff4a927e3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:21.012090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f26983-c84e-4bd9-b07b-70aff4a927e3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:21.038289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:21.047621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1731b1-237e-4db4-bed3-eb181c1181a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:21.048236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1731b1-237e-4db4-bed3-eb181c1181a8 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:21.048517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1731b1-237e-4db4-bed3-eb181c1181a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:21.048789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1731b1-237e-4db4-bed3-eb181c1181a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:21.049438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1731b1-237e-4db4-bed3-eb181c1181a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:21.050928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1731b1-237e-4db4-bed3-eb181c1181a8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:21.080449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 23/46] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:22:20 2022] POST /identity/v3/projects => generated 313 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:22:22.604421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12f4f3-f80d-4090-8fe9-4590425a3464 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:22.604943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12f4f3-f80d-4090-8fe9-4590425a3464 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:22.605031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12f4f3-f80d-4090-8fe9-4590425a3464 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:22.606340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 24/47] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:22.611535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2f0a-cccd-4ed7-8d49-40b0ff42a0d3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:22.611946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2f0a-cccd-4ed7-8d49-40b0ff42a0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:22.612246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2f0a-cccd-4ed7-8d49-40b0ff42a0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:22.631985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cb0c2f0a-cccd-4ed7-8d49-40b0ff42a0d3 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:22.633354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb0c2f0a-cccd-4ed7-8d49-40b0ff42a0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:22.705718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 24/48] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:22.987890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12f4f3-f80d-4090-8fe9-4590425a3464 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:23.015448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aad132-71ca-4bef-aacf-1b1202e74dff None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:23.015895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aad132-71ca-4bef-aacf-1b1202e74dff None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:23.016308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aad132-71ca-4bef-aacf-1b1202e74dff None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:23.017947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 25/49] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:22:23.025388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b9523-2ce2-4eda-b1a9-5db5abc24a3f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:23.025707 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b9523-2ce2-4eda-b1a9-5db5abc24a3f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:23.026098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b9523-2ce2-4eda-b1a9-5db5abc24a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:23.039275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:23.046408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d32b9523-2ce2-4eda-b1a9-5db5abc24a3f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:23.047609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d32b9523-2ce2-4eda-b1a9-5db5abc24a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:23.107916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12f4f3-f80d-4090-8fe9-4590425a3464 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:23.171599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaa221-647a-4502-a4e1-b07ea095b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:23.172624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaa221-647a-4502-a4e1-b07ea095b342 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:23.173083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaa221-647a-4502-a4e1-b07ea095b342 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:23.173543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaa221-647a-4502-a4e1-b07ea095b342 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:23.192667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaa221-647a-4502-a4e1-b07ea095b342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9dbaa221-647a-4502-a4e1-b07ea095b342 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:23.201633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:23.202659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:23.203422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:23.203996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 25/50] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:22:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:23.210638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaa221-647a-4502-a4e1-b07ea095b342 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:23.237632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 26/51] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:23.316477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaa221-647a-4502-a4e1-b07ea095b342 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:23.317987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:23.355505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74776469-6133-4f4b-be6a-4627e1769da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:23.356117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74776469-6133-4f4b-be6a-4627e1769da8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:23.356394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74776469-6133-4f4b-be6a-4627e1769da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:23.356663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74776469-6133-4f4b-be6a-4627e1769da8 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:23.357338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74776469-6133-4f4b-be6a-4627e1769da8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:23.357663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74776469-6133-4f4b-be6a-4627e1769da8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:23.359062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74776469-6133-4f4b-be6a-4627e1769da8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:23.367929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 26/52] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:23 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:23.373685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b9523-2ce2-4eda-b1a9-5db5abc24a3f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:23.416726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:23.476379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b9523-2ce2-4eda-b1a9-5db5abc24a3f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:23.514232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375e51d-b152-4c66-b732-aa592c1ab895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:23.514484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74776469-6133-4f4b-be6a-4627e1769da8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:23.514884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375e51d-b152-4c66-b732-aa592c1ab895 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:23.515174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375e51d-b152-4c66-b732-aa592c1ab895 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:23.515455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375e51d-b152-4c66-b732-aa592c1ab895 None admin] PATH_INFO: `/v3/users/d51a342066d64ae3b06abd4744cb8e8b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:23.516838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375e51d-b152-4c66-b732-aa592c1ab895 None admin] RBAC: Authorizing `identity:get_user(user_id=d51a342066d64ae3b06abd4744cb8e8b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:23.518339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375e51d-b152-4c66-b732-aa592c1ab895 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:23.519952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 27/53] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:23 2022] GET /identity/v3/users/d51a342066d64ae3b06abd4744cb8e8b => generated 248 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:23.526426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375e51d-b152-4c66-b732-aa592c1ab895 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:23.625253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74776469-6133-4f4b-be6a-4627e1769da8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:23.636173 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375e51d-b152-4c66-b732-aa592c1ab895 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:23.645330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:23.665151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a3ec5-748b-49d9-99e1-84528123fbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:23.665831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a3ec5-748b-49d9-99e1-84528123fbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:23.666113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a3ec5-748b-49d9-99e1-84528123fbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:23.666386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a3ec5-748b-49d9-99e1-84528123fbc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:23.667568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a3ec5-748b-49d9-99e1-84528123fbc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:23.669031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a3ec5-748b-49d9-99e1-84528123fbc0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:23.670937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 27/54] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:23 2022] GET /identity/v3/domains/default => generated 200 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:23.675317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716506b-a6cd-419f-aaf7-863108e61cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:23.675962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716506b-a6cd-419f-aaf7-863108e61cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:23.676246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716506b-a6cd-419f-aaf7-863108e61cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:23.676523 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716506b-a6cd-419f-aaf7-863108e61cd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:23.677714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716506b-a6cd-419f-aaf7-863108e61cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:23.677881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a3ec5-748b-49d9-99e1-84528123fbc0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:23.679867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716506b-a6cd-419f-aaf7-863108e61cd9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:23.681484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 28/55] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:23 2022] GET /identity/v3/domains/default => generated 200 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:23.687425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716506b-a6cd-419f-aaf7-863108e61cd9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:23.712297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:23.799843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a3ec5-748b-49d9-99e1-84528123fbc0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:23.807739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716506b-a6cd-419f-aaf7-863108e61cd9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:23.837557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9a5a8-3b23-4675-9ecf-e0d1150fd02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:23.838198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9a5a8-3b23-4675-9ecf-e0d1150fd02e None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:23.838479 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9a5a8-3b23-4675-9ecf-e0d1150fd02e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:23.838759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9a5a8-3b23-4675-9ecf-e0d1150fd02e None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:23.839537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9a5a8-3b23-4675-9ecf-e0d1150fd02e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:23.840893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9a5a8-3b23-4675-9ecf-e0d1150fd02e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:23.848859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7377c-7889-4a26-a80e-dd0bd2da4f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:23.849483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7377c-7889-4a26-a80e-dd0bd2da4f8c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:23.849832 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7377c-7889-4a26-a80e-dd0bd2da4f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:23.850113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7377c-7889-4a26-a80e-dd0bd2da4f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:23.851330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7377c-7889-4a26-a80e-dd0bd2da4f8c None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.domain.id=default, user.id=d51a342066d64ae3b06abd4744cb8e8b) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:23.851652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7377c-7889-4a26-a80e-dd0bd2da4f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:23.853545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7377c-7889-4a26-a80e-dd0bd2da4f8c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:23.863167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 29/56] 10.222.0.100 () {56 vars in 1327 bytes} [Thu Sep 1 17:22:23 2022] GET /identity/v3/role_assignments?user.id=d51a342066d64ae3b06abd4744cb8e8b&scope.domain.id=default&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 532 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:23.878130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 28/57] 10.222.0.100 () {60 vars in 1145 bytes} [Thu Sep 1 17:22:23 2022] POST /identity/v3/projects => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:22:25.881766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cf20d-ad08-48c6-95e7-2a2ff9567448 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:25.882565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cf20d-ad08-48c6-95e7-2a2ff9567448 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:25.882565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cf20d-ad08-48c6-95e7-2a2ff9567448 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:25.883724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 30/58] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:25.889646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916b350-0126-4071-b378-eb9e72326c5c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:25.890814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916b350-0126-4071-b378-eb9e72326c5c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:25.891093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916b350-0126-4071-b378-eb9e72326c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:25.911099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e916b350-0126-4071-b378-eb9e72326c5c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:25.912255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e916b350-0126-4071-b378-eb9e72326c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:25.986834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 29/59] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:26.253011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cf20d-ad08-48c6-95e7-2a2ff9567448 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:26.297419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:26.355480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cf20d-ad08-48c6-95e7-2a2ff9567448 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:26.388366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ea4f1-1910-49e0-a716-b427bd398908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:26.388997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ea4f1-1910-49e0-a716-b427bd398908 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:26.389279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ea4f1-1910-49e0-a716-b427bd398908 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:26.389552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ea4f1-1910-49e0-a716-b427bd398908 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:26.391138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ea4f1-1910-49e0-a716-b427bd398908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:26.392599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ea4f1-1910-49e0-a716-b427bd398908 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:26.394217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 31/60] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:26 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:26.400883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916b350-0126-4071-b378-eb9e72326c5c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:26.416569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:26.504725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916b350-0126-4071-b378-eb9e72326c5c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:26.547518 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e37d55-4c50-4e8d-a6d6-a5d93c27077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:26.548261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e37d55-4c50-4e8d-a6d6-a5d93c27077b None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:26.548571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e37d55-4c50-4e8d-a6d6-a5d93c27077b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:26.548849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e37d55-4c50-4e8d-a6d6-a5d93c27077b None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:26.549528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e37d55-4c50-4e8d-a6d6-a5d93c27077b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:26.551033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e37d55-4c50-4e8d-a6d6-a5d93c27077b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:26.583079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 30/61] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:22:26 2022] POST /identity/v3/users => generated 276 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:22:29.745564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc64f-d8c4-47d2-bcb4-6352f29e92ae None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:29.746663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc64f-d8c4-47d2-bcb4-6352f29e92ae None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:29.747091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc64f-d8c4-47d2-bcb4-6352f29e92ae None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:29.748961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 32/62] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:29 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:29.755122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd68b8-edee-4a6f-a1b8-3f53ea220c4d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:29.755968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd68b8-edee-4a6f-a1b8-3f53ea220c4d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:29.757113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd68b8-edee-4a6f-a1b8-3f53ea220c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:29.786704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-77fd68b8-edee-4a6f-a1b8-3f53ea220c4d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:29.788848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-77fd68b8-edee-4a6f-a1b8-3f53ea220c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:29.859893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63886d56-b3cc-4b09-8cd7-3be58f5d88b9 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:29.860527 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63886d56-b3cc-4b09-8cd7-3be58f5d88b9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:29.861009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63886d56-b3cc-4b09-8cd7-3be58f5d88b9 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:29.862676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 33/63] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:29 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:22:29.869359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb871cd-7158-42fb-81e1-51db4ca7f9e7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:29.869994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb871cd-7158-42fb-81e1-51db4ca7f9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:29.870934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb871cd-7158-42fb-81e1-51db4ca7f9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:29.893291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:29.903965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 31/64] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:29.905296 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8fb871cd-7158-42fb-81e1-51db4ca7f9e7 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:29.906760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8fb871cd-7158-42fb-81e1-51db4ca7f9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:30.009008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 34/65] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:30.133475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2aa56-1889-4528-a8ed-929d99337ae9 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:30.134157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2aa56-1889-4528-a8ed-929d99337ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:30.134643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2aa56-1889-4528-a8ed-929d99337ae9 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:30.136216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 32/66] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:30.141630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8a3e3-d17c-4413-a17b-9e89d469f0c4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:30.142211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8a3e3-d17c-4413-a17b-9e89d469f0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:30.142691 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8a3e3-d17c-4413-a17b-9e89d469f0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:30.166746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-84b8a3e3-d17c-4413-a17b-9e89d469f0c4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:30.168367 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-84b8a3e3-d17c-4413-a17b-9e89d469f0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:30.206136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:30.268111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 35/67] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:30.360581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2aa56-1889-4528-a8ed-929d99337ae9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:30.372790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3203-77e3-4fff-8eb3-ba33c9dfe2f7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:30.375846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3203-77e3-4fff-8eb3-ba33c9dfe2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:30.376361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3203-77e3-4fff-8eb3-ba33c9dfe2f7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:30.378052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 36/68] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:30 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:30.385322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463d038-3259-4b70-adad-5f46356abaaf None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:30.386073 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463d038-3259-4b70-adad-5f46356abaaf None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:30.386550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463d038-3259-4b70-adad-5f46356abaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:30.409237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f463d038-3259-4b70-adad-5f46356abaaf None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:30.411025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f463d038-3259-4b70-adad-5f46356abaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:30.503773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2aa56-1889-4528-a8ed-929d99337ae9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:30.505320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 37/69] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:30.507827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463d038-3259-4b70-adad-5f46356abaaf None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:30.560275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837be10-0f03-42cc-a337-4c29eea2fddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:30.561642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837be10-0f03-42cc-a337-4c29eea2fddb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:30.562427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837be10-0f03-42cc-a337-4c29eea2fddb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:30.563057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837be10-0f03-42cc-a337-4c29eea2fddb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:30.588551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837be10-0f03-42cc-a337-4c29eea2fddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2837be10-0f03-42cc-a337-4c29eea2fddb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:30.594056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:30.594961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:30.595755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:30.601684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 33/70] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:22:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:30.612610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837be10-0f03-42cc-a337-4c29eea2fddb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:30.679350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:30.685239 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463d038-3259-4b70-adad-5f46356abaaf None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:30.717704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:30.747656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1265a-7431-4c7d-a138-7a59eca01cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:30.748697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1265a-7431-4c7d-a138-7a59eca01cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:30.749162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1265a-7431-4c7d-a138-7a59eca01cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:30.750026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1265a-7431-4c7d-a138-7a59eca01cc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:30.767651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1265a-7431-4c7d-a138-7a59eca01cc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-adb1265a-7431-4c7d-a138-7a59eca01cc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:30.777443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:30.779331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 38/71] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:30.780899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1265a-7431-4c7d-a138-7a59eca01cc8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:30.792567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837be10-0f03-42cc-a337-4c29eea2fddb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:30.836571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e83efb-f4e8-4a6d-ad9f-46ddf5578f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:30.837196 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e83efb-f4e8-4a6d-ad9f-46ddf5578f01 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:30.837475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e83efb-f4e8-4a6d-ad9f-46ddf5578f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:30.837823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e83efb-f4e8-4a6d-ad9f-46ddf5578f01 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:30.838508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e83efb-f4e8-4a6d-ad9f-46ddf5578f01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:30.838812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e83efb-f4e8-4a6d-ad9f-46ddf5578f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:30.840261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e83efb-f4e8-4a6d-ad9f-46ddf5578f01 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:30.852297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 34/72] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:30 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:30.853706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0172-1db6-4579-98c0-d1dbb96a4e3a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:30.854082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0172-1db6-4579-98c0-d1dbb96a4e3a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:30.854265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0172-1db6-4579-98c0-d1dbb96a4e3a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:30.855909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 35/73] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:30.856283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0172-1db6-4579-98c0-d1dbb96a4e3a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:30.921567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1265a-7431-4c7d-a138-7a59eca01cc8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:30.963644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b261f-528a-4858-b440-c467b08ae1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:30.964295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b261f-528a-4858-b440-c467b08ae1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:30.964579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b261f-528a-4858-b440-c467b08ae1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:30.964855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b261f-528a-4858-b440-c467b08ae1b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:30.977802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b261f-528a-4858-b440-c467b08ae1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e18b261f-528a-4858-b440-c467b08ae1b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:30.982800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:30.983741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 39/74] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:22:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:30.991306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b261f-528a-4858-b440-c467b08ae1b3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:30.992205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0172-1db6-4579-98c0-d1dbb96a4e3a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:31.008939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:31.029886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:31.051007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956c151-ae25-44a3-96ca-76f11c4d7a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:31.052260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956c151-ae25-44a3-96ca-76f11c4d7a5e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.052747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956c151-ae25-44a3-96ca-76f11c4d7a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.053235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956c151-ae25-44a3-96ca-76f11c4d7a5e None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.054717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956c151-ae25-44a3-96ca-76f11c4d7a5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:31.055208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956c151-ae25-44a3-96ca-76f11c4d7a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:31.057783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956c151-ae25-44a3-96ca-76f11c4d7a5e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:31.074857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 36/75] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:30 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:31.076381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956c151-ae25-44a3-96ca-76f11c4d7a5e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:31.111985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b261f-528a-4858-b440-c467b08ae1b3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:31.171461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1d2de-1d90-4630-8486-52f867262d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:31.172106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1d2de-1d90-4630-8486-52f867262d57 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.172389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1d2de-1d90-4630-8486-52f867262d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.172661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1d2de-1d90-4630-8486-52f867262d57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.184473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1d2de-1d90-4630-8486-52f867262d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-cee1d2de-1d90-4630-8486-52f867262d57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:31.189376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:31.190350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 40/76] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:31.197121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d286e-dba5-4d12-8dbb-24f212d03595 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.197865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d286e-dba5-4d12-8dbb-24f212d03595 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.197865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d286e-dba5-4d12-8dbb-24f212d03595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.213271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-763d286e-dba5-4d12-8dbb-24f212d03595 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:31.214568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-763d286e-dba5-4d12-8dbb-24f212d03595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:31.215995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956c151-ae25-44a3-96ca-76f11c4d7a5e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:31.253416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110a973-e5d1-453c-84a3-9b2067a19e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:31.254078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110a973-e5d1-453c-84a3-9b2067a19e77 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.254357 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110a973-e5d1-453c-84a3-9b2067a19e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.254634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110a973-e5d1-453c-84a3-9b2067a19e77 None admin] PATH_INFO: `/v3/users/d51a342066d64ae3b06abd4744cb8e8b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.255790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110a973-e5d1-453c-84a3-9b2067a19e77 None admin] RBAC: Authorizing `identity:get_user(user_id=d51a342066d64ae3b06abd4744cb8e8b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:31.258239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110a973-e5d1-453c-84a3-9b2067a19e77 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:31.259898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 37/77] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:31 2022] GET /identity/v3/users/d51a342066d64ae3b06abd4744cb8e8b => generated 248 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:31.260700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110a973-e5d1-453c-84a3-9b2067a19e77 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:31.298109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 41/78] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:31.298946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d286e-dba5-4d12-8dbb-24f212d03595 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:31.327136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:31.342126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:31.421795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110a973-e5d1-453c-84a3-9b2067a19e77 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:31.447145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d286e-dba5-4d12-8dbb-24f212d03595 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:31.462254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c654e-a3df-40db-b1ac-7257d866f614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:31.462988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c654e-a3df-40db-b1ac-7257d866f614 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.463161 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c654e-a3df-40db-b1ac-7257d866f614 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.463428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c654e-a3df-40db-b1ac-7257d866f614 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.464218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c654e-a3df-40db-b1ac-7257d866f614 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:31.464526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c654e-a3df-40db-b1ac-7257d866f614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:31.466111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c654e-a3df-40db-b1ac-7257d866f614 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:31.478369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 38/79] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:22:31 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:31.479263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c654e-a3df-40db-b1ac-7257d866f614 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:31.484479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77c047-d236-4195-8676-c1b06a07aade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:31.485115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77c047-d236-4195-8676-c1b06a07aade None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.485395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77c047-d236-4195-8676-c1b06a07aade None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.485695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77c047-d236-4195-8676-c1b06a07aade None admin] PATH_INFO: `/v3/users/400981e92abc4ccea32372bcc765cfa8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.503260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77c047-d236-4195-8676-c1b06a07aade None admin] RBAC: Authorizing `identity:get_user(user_id=400981e92abc4ccea32372bcc765cfa8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:31.504760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77c047-d236-4195-8676-c1b06a07aade None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:31.507388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 42/80] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:31 2022] GET /identity/v3/users/400981e92abc4ccea32372bcc765cfa8 => generated 276 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:31.508236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77c047-d236-4195-8676-c1b06a07aade None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:31.590649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:31.614148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c654e-a3df-40db-b1ac-7257d866f614 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:31.631892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:31.637083 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77c047-d236-4195-8676-c1b06a07aade None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:31.654082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf54a6-17d1-4e22-a963-57edfdd5171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:31.655202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf54a6-17d1-4e22-a963-57edfdd5171f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.655202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf54a6-17d1-4e22-a963-57edfdd5171f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.655908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf54a6-17d1-4e22-a963-57edfdd5171f None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.656129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf54a6-17d1-4e22-a963-57edfdd5171f None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:31.656437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf54a6-17d1-4e22-a963-57edfdd5171f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:31.657913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf54a6-17d1-4e22-a963-57edfdd5171f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:31.667690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 39/81] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:31 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:31.668368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf54a6-17d1-4e22-a963-57edfdd5171f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:31.677518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216a79b-8f6f-4155-a4d5-c98a7fbb59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:31.678727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216a79b-8f6f-4155-a4d5-c98a7fbb59d5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.678990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216a79b-8f6f-4155-a4d5-c98a7fbb59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.679262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216a79b-8f6f-4155-a4d5-c98a7fbb59d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.755126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216a79b-8f6f-4155-a4d5-c98a7fbb59d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:31.757717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216a79b-8f6f-4155-a4d5-c98a7fbb59d5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:31.778529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf54a6-17d1-4e22-a963-57edfdd5171f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:31.815846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69207571-0bfa-44d2-9a8b-623bd4a21c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:31.816610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69207571-0bfa-44d2-9a8b-623bd4a21c6a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.816766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69207571-0bfa-44d2-9a8b-623bd4a21c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.817069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69207571-0bfa-44d2-9a8b-623bd4a21c6a None admin] PATH_INFO: `/v3/users/400981e92abc4ccea32372bcc765cfa8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.818328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69207571-0bfa-44d2-9a8b-623bd4a21c6a None admin] RBAC: Authorizing `identity:get_user(user_id=400981e92abc4ccea32372bcc765cfa8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:31.820386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69207571-0bfa-44d2-9a8b-623bd4a21c6a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:31.822048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 40/82] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:31 2022] GET /identity/v3/users/400981e92abc4ccea32372bcc765cfa8 => generated 276 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:31.826345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69207571-0bfa-44d2-9a8b-623bd4a21c6a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:31.833967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:31.872861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 43/83] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:31.873680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216a79b-8f6f-4155-a4d5-c98a7fbb59d5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:31.876503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:31.935214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69207571-0bfa-44d2-9a8b-623bd4a21c6a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:31.971910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fe102-aadc-4ffb-b5aa-c1865225eefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:31.972524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fe102-aadc-4ffb-b5aa-c1865225eefa None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:31.972818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fe102-aadc-4ffb-b5aa-c1865225eefa None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:31.973089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fe102-aadc-4ffb-b5aa-c1865225eefa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:31.984638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216a79b-8f6f-4155-a4d5-c98a7fbb59d5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:32.029709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858df8a4-a746-4889-a8b0-cbb550534b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:32.030352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858df8a4-a746-4889-a8b0-cbb550534b16 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:32.030634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858df8a4-a746-4889-a8b0-cbb550534b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:32.030908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858df8a4-a746-4889-a8b0-cbb550534b16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:32.032047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858df8a4-a746-4889-a8b0-cbb550534b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:32.033515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858df8a4-a746-4889-a8b0-cbb550534b16 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:32.035136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 44/84] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:31 2022] GET /identity/v3/domains/default => generated 200 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:32.036005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858df8a4-a746-4889-a8b0-cbb550534b16 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:32.049349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582fe102-aadc-4ffb-b5aa-c1865225eefa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:32.050872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582fe102-aadc-4ffb-b5aa-c1865225eefa None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:32.072437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:32.099370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:32.141255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858df8a4-a746-4889-a8b0-cbb550534b16 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:32.158439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 41/85] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:32.159311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fe102-aadc-4ffb-b5aa-c1865225eefa None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:32.182502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e67f84-7141-4465-9976-76f6b117bcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:32.183136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e67f84-7141-4465-9976-76f6b117bcb8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:32.183416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e67f84-7141-4465-9976-76f6b117bcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:32.183687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e67f84-7141-4465-9976-76f6b117bcb8 None admin] PATH_INFO: `/v3/users/400981e92abc4ccea32372bcc765cfa8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:32.184840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e67f84-7141-4465-9976-76f6b117bcb8 None admin] RBAC: Authorizing `identity:get_user(user_id=400981e92abc4ccea32372bcc765cfa8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:32.186320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e67f84-7141-4465-9976-76f6b117bcb8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:32.187908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 45/86] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:32 2022] GET /identity/v3/users/400981e92abc4ccea32372bcc765cfa8 => generated 276 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:32.188677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e67f84-7141-4465-9976-76f6b117bcb8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:32.276970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fe102-aadc-4ffb-b5aa-c1865225eefa None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:32.301463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e67f84-7141-4465-9976-76f6b117bcb8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:32.303675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:32.321715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81676430-7958-4b26-b379-dba6b5eb1c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:32.322357 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81676430-7958-4b26-b379-dba6b5eb1c84 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:32.323074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81676430-7958-4b26-b379-dba6b5eb1c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:32.323352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81676430-7958-4b26-b379-dba6b5eb1c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:32.340169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:32.351853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f454e-b280-489c-b566-5236b733d16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:32.352579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f454e-b280-489c-b566-5236b733d16f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:32.352872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f454e-b280-489c-b566-5236b733d16f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:32.353241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f454e-b280-489c-b566-5236b733d16f None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:32.354814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f454e-b280-489c-b566-5236b733d16f None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:32.356295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f454e-b280-489c-b566-5236b733d16f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:32.358054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 46/87] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:32 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:32.358841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f454e-b280-489c-b566-5236b733d16f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:32.413048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81676430-7958-4b26-b379-dba6b5eb1c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:32.415985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81676430-7958-4b26-b379-dba6b5eb1c84 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:32.477843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f454e-b280-489c-b566-5236b733d16f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:32.514931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71d197-feeb-45ee-86ce-e0cc66a88e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:32.515535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71d197-feeb-45ee-86ce-e0cc66a88e51 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:32.515816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71d197-feeb-45ee-86ce-e0cc66a88e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:32.516090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71d197-feeb-45ee-86ce-e0cc66a88e51 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:32.516787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71d197-feeb-45ee-86ce-e0cc66a88e51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:32.518222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71d197-feeb-45ee-86ce-e0cc66a88e51 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:32.534788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 42/88] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:32 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:32.534788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81676430-7958-4b26-b379-dba6b5eb1c84 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:32.551646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 47/89] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:22:32 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:22:32.552501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71d197-feeb-45ee-86ce-e0cc66a88e51 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:32.594088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:32.618598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:32.695586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81676430-7958-4b26-b379-dba6b5eb1c84 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:32.720542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71d197-feeb-45ee-86ce-e0cc66a88e51 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:32.754475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e89015-bf8e-4c79-950c-b4d8f4ad3f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:32.755564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e89015-bf8e-4c79-950c-b4d8f4ad3f15 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:32.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e89015-bf8e-4c79-950c-b4d8f4ad3f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:32.756762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e89015-bf8e-4c79-950c-b4d8f4ad3f15 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:32.758866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e89015-bf8e-4c79-950c-b4d8f4ad3f15 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:32.760762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e89015-bf8e-4c79-950c-b4d8f4ad3f15 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:32.763338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 43/90] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:32 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:32.771468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e89015-bf8e-4c79-950c-b4d8f4ad3f15 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:32.778544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f7111-e0d4-4ece-8aa6-8972988ef277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:32.778544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f7111-e0d4-4ece-8aa6-8972988ef277 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:32.778544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f7111-e0d4-4ece-8aa6-8972988ef277 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:32.778544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f7111-e0d4-4ece-8aa6-8972988ef277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:32.878440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f7111-e0d4-4ece-8aa6-8972988ef277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:32.884494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f7111-e0d4-4ece-8aa6-8972988ef277 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:32.931253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:32.936743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e89015-bf8e-4c79-950c-b4d8f4ad3f15 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:32.962421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:33.005764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab101e02-4811-4c0a-a713-5295d938b59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:33.007666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab101e02-4811-4c0a-a713-5295d938b59a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:33.008280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab101e02-4811-4c0a-a713-5295d938b59a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:33.008891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab101e02-4811-4c0a-a713-5295d938b59a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:33.010275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab101e02-4811-4c0a-a713-5295d938b59a None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=6fc37cae8bab49beb9244a40222ffda1, user.id=d51a342066d64ae3b06abd4744cb8e8b) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:33.010959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab101e02-4811-4c0a-a713-5295d938b59a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:33.014173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab101e02-4811-4c0a-a713-5295d938b59a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:33.025274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 44/91] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:32 2022] GET /identity/v3/role_assignments?user.id=d51a342066d64ae3b06abd4744cb8e8b&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 260 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:33.026974 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab101e02-4811-4c0a-a713-5295d938b59a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:33.077078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 48/92] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:32 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:33.077851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f7111-e0d4-4ece-8aa6-8972988ef277 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:33.219976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab101e02-4811-4c0a-a713-5295d938b59a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:33.278355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc53bfc-6e0a-45c6-8e43-f06d9e75cb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:33.279031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc53bfc-6e0a-45c6-8e43-f06d9e75cb6e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:33.279319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc53bfc-6e0a-45c6-8e43-f06d9e75cb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:33.279603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc53bfc-6e0a-45c6-8e43-f06d9e75cb6e None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:33.280863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f7111-e0d4-4ece-8aa6-8972988ef277 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:33.281077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc53bfc-6e0a-45c6-8e43-f06d9e75cb6e None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:33.282399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc53bfc-6e0a-45c6-8e43-f06d9e75cb6e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:33.284127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 45/93] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:33 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:33.285033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc53bfc-6e0a-45c6-8e43-f06d9e75cb6e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:33.354973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:33.367117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221bef2-ea2a-48b9-aa2e-0dcdb662d87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:33.367756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221bef2-ea2a-48b9-aa2e-0dcdb662d87b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:33.368034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221bef2-ea2a-48b9-aa2e-0dcdb662d87b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:33.368304 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221bef2-ea2a-48b9-aa2e-0dcdb662d87b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:33.369052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221bef2-ea2a-48b9-aa2e-0dcdb662d87b None admin] RBAC: Adding query filter params (role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=6fc37cae8bab49beb9244a40222ffda1, user.id=400981e92abc4ccea32372bcc765cfa8) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:33.369360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221bef2-ea2a-48b9-aa2e-0dcdb662d87b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:33.370879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221bef2-ea2a-48b9-aa2e-0dcdb662d87b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:33.384372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 49/94] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:33 2022] GET /identity/v3/role_assignments?user.id=400981e92abc4ccea32372bcc765cfa8&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=90949fa670c2407ca92b16156e52f47f => generated 260 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:33.385047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221bef2-ea2a-48b9-aa2e-0dcdb662d87b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:33.413999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:33.546077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221bef2-ea2a-48b9-aa2e-0dcdb662d87b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:33.582553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc53bfc-6e0a-45c6-8e43-f06d9e75cb6e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:33.586708 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fb700-a697-4bd1-8c5b-6296fbf9e4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:33.587360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fb700-a697-4bd1-8c5b-6296fbf9e4ea None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:33.587644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fb700-a697-4bd1-8c5b-6296fbf9e4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:33.587922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fb700-a697-4bd1-8c5b-6296fbf9e4ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:33.588839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fb700-a697-4bd1-8c5b-6296fbf9e4ea None admin] RBAC: Adding query filter params (role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=6fc37cae8bab49beb9244a40222ffda1, user.id=400981e92abc4ccea32372bcc765cfa8) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:33.589155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fb700-a697-4bd1-8c5b-6296fbf9e4ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:33.590723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fb700-a697-4bd1-8c5b-6296fbf9e4ea None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:33.597277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 50/95] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:33 2022] GET /identity/v3/role_assignments?user.id=400981e92abc4ccea32372bcc765cfa8&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=5dc7056ddc154b81939a17474a64f546 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:33.684601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46a0aa-f299-4eb1-b8d2-2d67d70d1fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:33.685306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46a0aa-f299-4eb1-b8d2-2d67d70d1fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:33.685742 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46a0aa-f299-4eb1-b8d2-2d67d70d1fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:33.686041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46a0aa-f299-4eb1-b8d2-2d67d70d1fd1 None admin] PATH_INFO: `/v3/projects/af4040b3388e46e2b2c64306f872bbf4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:33.690564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46a0aa-f299-4eb1-b8d2-2d67d70d1fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=af4040b3388e46e2b2c64306f872bbf4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:33.692103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46a0aa-f299-4eb1-b8d2-2d67d70d1fd1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:33.694650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 46/96] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:33 2022] GET /identity/v3/projects/af4040b3388e46e2b2c64306f872bbf4 => generated 313 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:33.701095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fb700-a697-4bd1-8c5b-6296fbf9e4ea None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:33.810384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:33.867555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fb700-a697-4bd1-8c5b-6296fbf9e4ea None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:33.908805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f3702-8b38-478a-8801-d2bffbb96a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:33.909796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f3702-8b38-478a-8801-d2bffbb96a02 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:33.910255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f3702-8b38-478a-8801-d2bffbb96a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:33.910679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f3702-8b38-478a-8801-d2bffbb96a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:33.911672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f3702-8b38-478a-8801-d2bffbb96a02 None admin] RBAC: Adding query filter params (role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=af4040b3388e46e2b2c64306f872bbf4, user.id=400981e92abc4ccea32372bcc765cfa8) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:33.912134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f3702-8b38-478a-8801-d2bffbb96a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:33.914679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f3702-8b38-478a-8801-d2bffbb96a02 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:33.921474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 51/97] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:33 2022] GET /identity/v3/role_assignments?user.id=400981e92abc4ccea32372bcc765cfa8&scope.project.id=af4040b3388e46e2b2c64306f872bbf4&role.id=90949fa670c2407ca92b16156e52f47f => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:35.535414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b163d-913b-4ac4-a51a-03fa5ab8c7d9 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:35.535956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b163d-913b-4ac4-a51a-03fa5ab8c7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:35.535956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b163d-913b-4ac4-a51a-03fa5ab8c7d9 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:35.538402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 47/98] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:35 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:35.544037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56b1af-e147-49cd-b024-f9f432b9d397 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:35.544625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56b1af-e147-49cd-b024-f9f432b9d397 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:35.544625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56b1af-e147-49cd-b024-f9f432b9d397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:35.567592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-db56b1af-e147-49cd-b024-f9f432b9d397 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:35.568965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-db56b1af-e147-49cd-b024-f9f432b9d397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:35.626075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96365e4-2db4-4e04-ac30-4b536264bd6c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:35.626377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96365e4-2db4-4e04-ac30-4b536264bd6c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:35.626647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96365e4-2db4-4e04-ac30-4b536264bd6c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:35.628005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 48/99] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:22:35.633971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292681fa-8d8b-4aab-b258-f6bb33e77d89 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:35.634257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292681fa-8d8b-4aab-b258-f6bb33e77d89 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:35.634536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292681fa-8d8b-4aab-b258-f6bb33e77d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:35.666859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-292681fa-8d8b-4aab-b258-f6bb33e77d89 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:35.668101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-292681fa-8d8b-4aab-b258-f6bb33e77d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:35.718001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 52/100] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:35.803993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 49/101] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:35.868770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ebe65-264e-4aa2-846f-c8479ed13cd5 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:35.869094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ebe65-264e-4aa2-846f-c8479ed13cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:35.869343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ebe65-264e-4aa2-846f-c8479ed13cd5 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:35.870973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 53/102] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:22:35.876469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0868ba-1798-4a37-8467-4754379cb959 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:35.876765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0868ba-1798-4a37-8467-4754379cb959 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:35.877051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0868ba-1798-4a37-8467-4754379cb959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:35.903833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bc0868ba-1798-4a37-8467-4754379cb959 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:35.905074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bc0868ba-1798-4a37-8467-4754379cb959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:35.963019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:36.011536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 50/103] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:36.143841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ebe65-264e-4aa2-846f-c8479ed13cd5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:36.221732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:36.294822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ebe65-264e-4aa2-846f-c8479ed13cd5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:36.301610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0868ba-1798-4a37-8467-4754379cb959 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:36.400376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a049-1c49-44e6-b6d0-6cd423d2b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:36.401124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a049-1c49-44e6-b6d0-6cd423d2b553 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:36.401415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a049-1c49-44e6-b6d0-6cd423d2b553 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:36.401719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a049-1c49-44e6-b6d0-6cd423d2b553 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:36.402932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74a049-1c49-44e6-b6d0-6cd423d2b553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:36.404577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74a049-1c49-44e6-b6d0-6cd423d2b553 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:36.409709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 54/104] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:36 2022] GET /identity/v3/domains/default => generated 200 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:36.411958 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a049-1c49-44e6-b6d0-6cd423d2b553 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:36.538149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0868ba-1798-4a37-8467-4754379cb959 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:36.601229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289bc3-afc9-4bc8-bf00-2cf91136c4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:36.602007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289bc3-afc9-4bc8-bf00-2cf91136c4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:36.602291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289bc3-afc9-4bc8-bf00-2cf91136c4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:36.602565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289bc3-afc9-4bc8-bf00-2cf91136c4d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:36.628130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a049-1c49-44e6-b6d0-6cd423d2b553 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:36.630903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43289bc3-afc9-4bc8-bf00-2cf91136c4d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-43289bc3-afc9-4bc8-bf00-2cf91136c4d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:36.636084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:36.637028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:22:36.637827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:36.644667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 51/105] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:22:36.645742 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289bc3-afc9-4bc8-bf00-2cf91136c4d6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:36.675682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:36.705111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:36.707122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519e487-9a45-480f-b478-4f418afd52d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:36.707793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519e487-9a45-480f-b478-4f418afd52d5 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:36.708079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519e487-9a45-480f-b478-4f418afd52d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:36.708356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519e487-9a45-480f-b478-4f418afd52d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:36.709053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519e487-9a45-480f-b478-4f418afd52d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:36.711359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519e487-9a45-480f-b478-4f418afd52d5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:36.768270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 55/106] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:22:36 2022] POST /identity/v3/projects => generated 303 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:22:36.769596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519e487-9a45-480f-b478-4f418afd52d5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:36.871283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289bc3-afc9-4bc8-bf00-2cf91136c4d6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:36.919271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519e487-9a45-480f-b478-4f418afd52d5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:36.930933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbffae2-1fc8-4217-9b22-3df75117363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:36.932556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbffae2-1fc8-4217-9b22-3df75117363c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:36.932861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbffae2-1fc8-4217-9b22-3df75117363c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:36.933147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbffae2-1fc8-4217-9b22-3df75117363c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:36.946810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbffae2-1fc8-4217-9b22-3df75117363c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-7dbffae2-1fc8-4217-9b22-3df75117363c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:36.952158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:36.953053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:36.953852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 52/107] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:22:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:36.955003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fb4fb-08f6-417c-a6ae-a06c1c7d98a6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:36.955003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fb4fb-08f6-417c-a6ae-a06c1c7d98a6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:36.955227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fb4fb-08f6-417c-a6ae-a06c1c7d98a6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:36.957147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 53/108] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:36.961150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469edea-70f1-40b5-9433-71ccb8bf8e52 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:36.961150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469edea-70f1-40b5-9433-71ccb8bf8e52 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:36.961150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469edea-70f1-40b5-9433-71ccb8bf8e52 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:36.961150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 54/109] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:36.966236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469edea-70f1-40b5-9433-71ccb8bf8e52 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:36.988124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8e0fe-2f21-4daf-a9df-307509731ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:36.988915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8e0fe-2f21-4daf-a9df-307509731ede None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:36.989144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8e0fe-2f21-4daf-a9df-307509731ede None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:36.989429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8e0fe-2f21-4daf-a9df-307509731ede None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:36.990587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8e0fe-2f21-4daf-a9df-307509731ede None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:36.990587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8e0fe-2f21-4daf-a9df-307509731ede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:36.992140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8e0fe-2f21-4daf-a9df-307509731ede None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:37.002748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 56/110] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:36 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:37.004135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dde36-9f32-4794-acd4-baece646b593 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:37.004442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dde36-9f32-4794-acd4-baece646b593 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:37.004725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dde36-9f32-4794-acd4-baece646b593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:37.031246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1e1dde36-9f32-4794-acd4-baece646b593 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:37.033559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1e1dde36-9f32-4794-acd4-baece646b593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:37.073203 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:37.104214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:37.115772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469edea-70f1-40b5-9433-71ccb8bf8e52 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:37.154224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 57/111] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:37.162943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ae2cb-3181-4a38-9b42-459b9a3edc3d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:37.163359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ae2cb-3181-4a38-9b42-459b9a3edc3d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:37.163542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ae2cb-3181-4a38-9b42-459b9a3edc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:37.166515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f53441-191c-4651-bc10-58e65ffd46bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:37.167552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53441-191c-4651-bc10-58e65ffd46bf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:37.168095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53441-191c-4651-bc10-58e65ffd46bf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:37.168606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53441-191c-4651-bc10-58e65ffd46bf None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:37.170481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f53441-191c-4651-bc10-58e65ffd46bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:37.171199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f53441-191c-4651-bc10-58e65ffd46bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:37.173901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f53441-191c-4651-bc10-58e65ffd46bf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:37.193622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 55/112] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:36 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:37.195810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f53441-191c-4651-bc10-58e65ffd46bf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:37.198907 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1a5ae2cb-3181-4a38-9b42-459b9a3edc3d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:37.200248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1a5ae2cb-3181-4a38-9b42-459b9a3edc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:37.343322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 58/113] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:37.344969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ae2cb-3181-4a38-9b42-459b9a3edc3d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:37.363973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f53441-191c-4651-bc10-58e65ffd46bf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:37.414983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc09a4f-5d69-4046-9e42-f474617e9bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:37.416653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc09a4f-5d69-4046-9e42-f474617e9bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:37.417452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc09a4f-5d69-4046-9e42-f474617e9bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:37.418075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc09a4f-5d69-4046-9e42-f474617e9bc5 None admin] PATH_INFO: `/v3/users/4fa10b3cc8464783913969ccc8c0c988` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:37.439855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc09a4f-5d69-4046-9e42-f474617e9bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=4fa10b3cc8464783913969ccc8c0c988)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:37.441729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc09a4f-5d69-4046-9e42-f474617e9bc5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:37.445206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 56/114] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:37 2022] GET /identity/v3/users/4fa10b3cc8464783913969ccc8c0c988 => generated 290 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:37.447173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09375382-4835-49b4-871f-3a7832c79c0a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:37.447808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09375382-4835-49b4-871f-3a7832c79c0a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:37.448272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09375382-4835-49b4-871f-3a7832c79c0a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:37.450857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 57/115] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:37 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:37.451873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09375382-4835-49b4-871f-3a7832c79c0a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:37.471747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:37.474380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:37.486711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ae2cb-3181-4a38-9b42-459b9a3edc3d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:37.547540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad38bf-fae6-4635-8826-4ac60fad24e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:37.549186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad38bf-fae6-4635-8826-4ac60fad24e5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:37.549478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad38bf-fae6-4635-8826-4ac60fad24e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:37.549788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad38bf-fae6-4635-8826-4ac60fad24e5 None admin] PATH_INFO: `/v3/users/d51a342066d64ae3b06abd4744cb8e8b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:37.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad38bf-fae6-4635-8826-4ac60fad24e5 None admin] RBAC: Authorizing `identity:get_user(user_id=d51a342066d64ae3b06abd4744cb8e8b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:37.552452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad38bf-fae6-4635-8826-4ac60fad24e5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:37.555501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 59/116] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:37 2022] GET /identity/v3/users/d51a342066d64ae3b06abd4744cb8e8b => generated 248 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:37.556705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd367d6-fd1e-40c2-8f79-442721e8a636 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:37.556989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd367d6-fd1e-40c2-8f79-442721e8a636 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:37.557261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd367d6-fd1e-40c2-8f79-442721e8a636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:37.581910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6dd367d6-fd1e-40c2-8f79-442721e8a636 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:37.583131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6dd367d6-fd1e-40c2-8f79-442721e8a636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:37.595813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09375382-4835-49b4-871f-3a7832c79c0a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:37.636352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e73c4-074f-4400-8233-ea0775a6d788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:37.637025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e73c4-074f-4400-8233-ea0775a6d788 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:37.637255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e73c4-074f-4400-8233-ea0775a6d788 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:37.637538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e73c4-074f-4400-8233-ea0775a6d788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:37.687261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 60/117] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:37.694981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd367d6-fd1e-40c2-8f79-442721e8a636 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:37.736779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e73c4-074f-4400-8233-ea0775a6d788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:37.738619 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e73c4-074f-4400-8233-ea0775a6d788 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:37.774065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:37.801791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:37.816726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd367d6-fd1e-40c2-8f79-442721e8a636 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:37.861337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b361e9-b688-4864-9004-1e1e09f158b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:37.862040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b361e9-b688-4864-9004-1e1e09f158b4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:37.862320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b361e9-b688-4864-9004-1e1e09f158b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:37.862595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b361e9-b688-4864-9004-1e1e09f158b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:37.868824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 58/118] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:37.870153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e73c4-074f-4400-8233-ea0775a6d788 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:37.882474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b361e9-b688-4864-9004-1e1e09f158b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d0b361e9-b688-4864-9004-1e1e09f158b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:37.887549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:37.888479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:37.889280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:37.894003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 61/119] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:37.894674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b361e9-b688-4864-9004-1e1e09f158b4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.001425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e73c4-074f-4400-8233-ea0775a6d788 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.006093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b361e9-b688-4864-9004-1e1e09f158b4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.015743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:38.049171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfb439-3f30-4971-a7b1-82f8a924893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.049849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfb439-3f30-4971-a7b1-82f8a924893c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.050136 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfb439-3f30-4971-a7b1-82f8a924893c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.050413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfb439-3f30-4971-a7b1-82f8a924893c None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.051197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c55c2-512a-4c4a-b3bb-d25628db7ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.051669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfb439-3f30-4971-a7b1-82f8a924893c None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:38.052023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c55c2-512a-4c4a-b3bb-d25628db7ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.052396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c55c2-512a-4c4a-b3bb-d25628db7ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.052846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c55c2-512a-4c4a-b3bb-d25628db7ba7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.053372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfb439-3f30-4971-a7b1-82f8a924893c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:38.060853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 59/120] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:37 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:38.060853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfb439-3f30-4971-a7b1-82f8a924893c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.070243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c55c2-512a-4c4a-b3bb-d25628db7ba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-302c55c2-512a-4c4a-b3bb-d25628db7ba7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:38.075326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:38.076230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:38.077014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:38.081249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 62/121] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:22:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:38.082503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c55c2-512a-4c4a-b3bb-d25628db7ba7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.096313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:38.197766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfb439-3f30-4971-a7b1-82f8a924893c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.208935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c55c2-512a-4c4a-b3bb-d25628db7ba7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.234665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1831b-4c58-45c6-8e8a-bd77f0ddff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.235306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1831b-4c58-45c6-8e8a-bd77f0ddff96 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.235591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1831b-4c58-45c6-8e8a-bd77f0ddff96 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.235871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1831b-4c58-45c6-8e8a-bd77f0ddff96 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.237029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1831b-4c58-45c6-8e8a-bd77f0ddff96 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:38.238533 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1831b-4c58-45c6-8e8a-bd77f0ddff96 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:38.240188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 60/122] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:38 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:38.241484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1831b-4c58-45c6-8e8a-bd77f0ddff96 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.253115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5498a-ad84-49e6-b49b-28a2b581749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.254148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5498a-ad84-49e6-b49b-28a2b581749b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.254593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5498a-ad84-49e6-b49b-28a2b581749b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.255012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5498a-ad84-49e6-b49b-28a2b581749b None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.255986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5498a-ad84-49e6-b49b-28a2b581749b None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:38.257037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5498a-ad84-49e6-b49b-28a2b581749b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:38.258762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5498a-ad84-49e6-b49b-28a2b581749b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:38.268744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 63/123] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:38 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:38.269863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5498a-ad84-49e6-b49b-28a2b581749b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.299089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:38.353609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:38.361069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1831b-4c58-45c6-8e8a-bd77f0ddff96 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.387265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5498a-ad84-49e6-b49b-28a2b581749b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.401230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06e470-dc1a-4a08-ac58-884ae0c8725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.401907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06e470-dc1a-4a08-ac58-884ae0c8725c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.402191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06e470-dc1a-4a08-ac58-884ae0c8725c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.402473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06e470-dc1a-4a08-ac58-884ae0c8725c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.414885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06e470-dc1a-4a08-ac58-884ae0c8725c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4d06e470-dc1a-4a08-ac58-884ae0c8725c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:38.420306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:38.421207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 61/124] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:38.426203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06e470-dc1a-4a08-ac58-884ae0c8725c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.440522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28017722-352e-4695-b057-220686cf55b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.441211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28017722-352e-4695-b057-220686cf55b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.441496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28017722-352e-4695-b057-220686cf55b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.441816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28017722-352e-4695-b057-220686cf55b4 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1/users/d51a342066d64ae3b06abd4744cb8e8b/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.443904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28017722-352e-4695-b057-220686cf55b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc37cae8bab49beb9244a40222ffda1, user_id=d51a342066d64ae3b06abd4744cb8e8b, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:38.446776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28017722-352e-4695-b057-220686cf55b4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:38.464195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 64/125] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:22:38 2022] PUT /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1/users/d51a342066d64ae3b06abd4744cb8e8b/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:22:38.464918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28017722-352e-4695-b057-220686cf55b4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.584897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06e470-dc1a-4a08-ac58-884ae0c8725c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.598709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:38.630235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28017722-352e-4695-b057-220686cf55b4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.643805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cce91-85e9-4bf4-b8d2-e1a71f695c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.644820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cce91-85e9-4bf4-b8d2-e1a71f695c9e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.645278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cce91-85e9-4bf4-b8d2-e1a71f695c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.645945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cce91-85e9-4bf4-b8d2-e1a71f695c9e None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.646423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cce91-85e9-4bf4-b8d2-e1a71f695c9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:38.646906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cce91-85e9-4bf4-b8d2-e1a71f695c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:38.648544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cce91-85e9-4bf4-b8d2-e1a71f695c9e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:38.665752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 62/126] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:22:38 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:38.665752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cce91-85e9-4bf4-b8d2-e1a71f695c9e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.681833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:38.697602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afecac9-0349-44e2-a321-66b97ccdd1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.697602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afecac9-0349-44e2-a321-66b97ccdd1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.697602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afecac9-0349-44e2-a321-66b97ccdd1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.697602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afecac9-0349-44e2-a321-66b97ccdd1c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.698502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afecac9-0349-44e2-a321-66b97ccdd1c8 None admin] RBAC: Adding query filter params (role.id=e272ecbda33d4b3e9308242d2d4ab302, scope.project.id=6fc37cae8bab49beb9244a40222ffda1, user.id=4fa10b3cc8464783913969ccc8c0c988) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:38.698502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afecac9-0349-44e2-a321-66b97ccdd1c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:38.698502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afecac9-0349-44e2-a321-66b97ccdd1c8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:38.703986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 65/127] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:38 2022] GET /identity/v3/role_assignments?user.id=4fa10b3cc8464783913969ccc8c0c988&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=e272ecbda33d4b3e9308242d2d4ab302 => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:38.704878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afecac9-0349-44e2-a321-66b97ccdd1c8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.840671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cce91-85e9-4bf4-b8d2-e1a71f695c9e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.887583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b617e-fa27-417e-914d-409dcac71a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.887583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b617e-fa27-417e-914d-409dcac71a10 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.887583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b617e-fa27-417e-914d-409dcac71a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.888240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b617e-fa27-417e-914d-409dcac71a10 None admin] PATH_INFO: `/v3/users/400981e92abc4ccea32372bcc765cfa8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.890687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b617e-fa27-417e-914d-409dcac71a10 None admin] RBAC: Authorizing `identity:get_user(user_id=400981e92abc4ccea32372bcc765cfa8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:38.892320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b617e-fa27-417e-914d-409dcac71a10 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:38.894582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 63/128] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:38 2022] GET /identity/v3/users/400981e92abc4ccea32372bcc765cfa8 => generated 276 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:38.902641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b617e-fa27-417e-914d-409dcac71a10 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:38.915379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afecac9-0349-44e2-a321-66b97ccdd1c8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:38.950618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:38.982346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d2c12-b816-4558-ac29-bef911b8d9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:38.982979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d2c12-b816-4558-ac29-bef911b8d9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:38.983672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d2c12-b816-4558-ac29-bef911b8d9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:38.983672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d2c12-b816-4558-ac29-bef911b8d9a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:38.984195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d2c12-b816-4558-ac29-bef911b8d9a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:38.984506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d2c12-b816-4558-ac29-bef911b8d9a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:38.986014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d2c12-b816-4558-ac29-bef911b8d9a1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:38.998455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 66/129] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:38 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:38.999074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d2c12-b816-4558-ac29-bef911b8d9a1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:39.021131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b617e-fa27-417e-914d-409dcac71a10 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:39.059542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f3a61-4460-4330-afd2-1b0fc2328f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:39.060530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f3a61-4460-4330-afd2-1b0fc2328f65 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.061005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f3a61-4460-4330-afd2-1b0fc2328f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.061496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f3a61-4460-4330-afd2-1b0fc2328f65 None admin] PATH_INFO: `/v3/users/400981e92abc4ccea32372bcc765cfa8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.062881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f3a61-4460-4330-afd2-1b0fc2328f65 None admin] RBAC: Authorizing `identity:get_user(user_id=400981e92abc4ccea32372bcc765cfa8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:39.064529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f3a61-4460-4330-afd2-1b0fc2328f65 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:39.066383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 64/130] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:38 2022] GET /identity/v3/users/400981e92abc4ccea32372bcc765cfa8 => generated 276 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:39.067351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f3a61-4460-4330-afd2-1b0fc2328f65 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:39.086118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:39.134340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d2c12-b816-4558-ac29-bef911b8d9a1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:39.173264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d3ddc-f330-4d7c-a43d-407173100143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:39.174212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d3ddc-f330-4d7c-a43d-407173100143 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.174503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d3ddc-f330-4d7c-a43d-407173100143 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.174795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d3ddc-f330-4d7c-a43d-407173100143 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.175999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d3ddc-f330-4d7c-a43d-407173100143 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:39.177466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d3ddc-f330-4d7c-a43d-407173100143 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:39.179116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f3a61-4460-4330-afd2-1b0fc2328f65 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:39.183753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 67/131] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:38 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:39.183753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1b6b1-a744-4316-8ea6-58f076edbf48 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.183753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1b6b1-a744-4316-8ea6-58f076edbf48 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.183753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1b6b1-a744-4316-8ea6-58f076edbf48 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.183753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 68/132] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:39 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:39.183753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1b6b1-a744-4316-8ea6-58f076edbf48 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:39.193555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:39.224091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8634e-65a1-4062-a08a-1b1b65a0aa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:39.225681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8634e-65a1-4062-a08a-1b1b65a0aa1f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.226428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8634e-65a1-4062-a08a-1b1b65a0aa1f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.226988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8634e-65a1-4062-a08a-1b1b65a0aa1f None admin] PATH_INFO: `/v3/users/400981e92abc4ccea32372bcc765cfa8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.228347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8634e-65a1-4062-a08a-1b1b65a0aa1f None admin] RBAC: Authorizing `identity:get_user(user_id=400981e92abc4ccea32372bcc765cfa8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:39.230069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8634e-65a1-4062-a08a-1b1b65a0aa1f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:39.231895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 65/133] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:39 2022] GET /identity/v3/users/400981e92abc4ccea32372bcc765cfa8 => generated 276 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:39.232879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8634e-65a1-4062-a08a-1b1b65a0aa1f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:39.300145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1b6b1-a744-4316-8ea6-58f076edbf48 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:39.326207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:39.338317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a5422-e6aa-4866-9e79-aa54a349824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:39.339186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a5422-e6aa-4866-9e79-aa54a349824c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.339472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a5422-e6aa-4866-9e79-aa54a349824c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.339751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a5422-e6aa-4866-9e79-aa54a349824c None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.341204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a5422-e6aa-4866-9e79-aa54a349824c None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:39.343407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a5422-e6aa-4866-9e79-aa54a349824c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:39.345769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 69/134] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:39 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:39.351578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e825fa1-da63-424f-a3fe-724f18c13729 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.351864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e825fa1-da63-424f-a3fe-724f18c13729 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.352136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e825fa1-da63-424f-a3fe-724f18c13729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.355472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8634e-65a1-4062-a08a-1b1b65a0aa1f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:39.374360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4e825fa1-da63-424f-a3fe-724f18c13729 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:39.375547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4e825fa1-da63-424f-a3fe-724f18c13729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:39.402305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a120fe2-274c-4886-b6ea-9b0aabb9a691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:39.402975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a120fe2-274c-4886-b6ea-9b0aabb9a691 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.403255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a120fe2-274c-4886-b6ea-9b0aabb9a691 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.404211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a120fe2-274c-4886-b6ea-9b0aabb9a691 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1/users/400981e92abc4ccea32372bcc765cfa8/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.407593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a120fe2-274c-4886-b6ea-9b0aabb9a691 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc37cae8bab49beb9244a40222ffda1, user_id=400981e92abc4ccea32372bcc765cfa8, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:39.409678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a120fe2-274c-4886-b6ea-9b0aabb9a691 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:39.430082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 66/135] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:22:39 2022] PUT /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1/users/400981e92abc4ccea32372bcc765cfa8/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:22:39.431480 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a120fe2-274c-4886-b6ea-9b0aabb9a691 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:39.477285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:39.524871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 70/136] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:39.525764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e825fa1-da63-424f-a3fe-724f18c13729 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:39.563230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a120fe2-274c-4886-b6ea-9b0aabb9a691 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:39.605950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a909d2-c75f-4b8e-9fad-6a41d4e3bcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:39.607757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a909d2-c75f-4b8e-9fad-6a41d4e3bcab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.608264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a909d2-c75f-4b8e-9fad-6a41d4e3bcab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.608744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a909d2-c75f-4b8e-9fad-6a41d4e3bcab None admin] PATH_INFO: `/v3/projects/af4040b3388e46e2b2c64306f872bbf4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.610293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a909d2-c75f-4b8e-9fad-6a41d4e3bcab None admin] RBAC: Authorizing `identity:get_project(project_id=af4040b3388e46e2b2c64306f872bbf4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:39.612350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a909d2-c75f-4b8e-9fad-6a41d4e3bcab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:39.614430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 67/137] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:39 2022] GET /identity/v3/projects/af4040b3388e46e2b2c64306f872bbf4 => generated 313 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:39.623764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a909d2-c75f-4b8e-9fad-6a41d4e3bcab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:39.746688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e825fa1-da63-424f-a3fe-724f18c13729 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:39.769632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a909d2-c75f-4b8e-9fad-6a41d4e3bcab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:39.772485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:39.777415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:39.792635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a38cd-2fd0-4c53-b164-8cb58d6805c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:39.793309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a38cd-2fd0-4c53-b164-8cb58d6805c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.793601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a38cd-2fd0-4c53-b164-8cb58d6805c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.793895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a38cd-2fd0-4c53-b164-8cb58d6805c4 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1/users/400981e92abc4ccea32372bcc765cfa8/roles/5dc7056ddc154b81939a17474a64f546` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.796474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a38cd-2fd0-4c53-b164-8cb58d6805c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc37cae8bab49beb9244a40222ffda1, user_id=400981e92abc4ccea32372bcc765cfa8, role_id=5dc7056ddc154b81939a17474a64f546)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:39.797967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a38cd-2fd0-4c53-b164-8cb58d6805c4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:39.817612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 71/138] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:22:39 2022] PUT /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1/users/400981e92abc4ccea32372bcc765cfa8/roles/5dc7056ddc154b81939a17474a64f546 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:22:39.828186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3fd99-036f-4b2c-b6fd-07f38e653569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:39.828933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3fd99-036f-4b2c-b6fd-07f38e653569 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:39.829213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3fd99-036f-4b2c-b6fd-07f38e653569 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:39.829550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3fd99-036f-4b2c-b6fd-07f38e653569 None admin] PATH_INFO: `/v3/projects/af4040b3388e46e2b2c64306f872bbf4/users/400981e92abc4ccea32372bcc765cfa8/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:39.831990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3fd99-036f-4b2c-b6fd-07f38e653569 None admin] RBAC: Authorizing `identity:create_grant(project_id=af4040b3388e46e2b2c64306f872bbf4, user_id=400981e92abc4ccea32372bcc765cfa8, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:39.833721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3fd99-036f-4b2c-b6fd-07f38e653569 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:39.854203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 68/139] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:22:39 2022] PUT /identity/v3/projects/af4040b3388e46e2b2c64306f872bbf4/users/400981e92abc4ccea32372bcc765cfa8/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:22:40.034835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a38cd-2fd0-4c53-b164-8cb58d6805c4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:40.201051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a38cd-2fd0-4c53-b164-8cb58d6805c4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:40.254377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a50d02-cf0f-4100-9c59-27c2e182c096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:40.255083 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a50d02-cf0f-4100-9c59-27c2e182c096 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:40.255414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a50d02-cf0f-4100-9c59-27c2e182c096 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:40.255690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a50d02-cf0f-4100-9c59-27c2e182c096 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:40.257850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a50d02-cf0f-4100-9c59-27c2e182c096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:40.259288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a50d02-cf0f-4100-9c59-27c2e182c096 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:40.260904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 72/140] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:40 2022] GET /identity/v3/domains/default => generated 200 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:40.267067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3fd99-036f-4b2c-b6fd-07f38e653569 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:40.416902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3fd99-036f-4b2c-b6fd-07f38e653569 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:40.469901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3bbc2-b550-4a17-a44f-f64fe9773354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:40.470919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3bbc2-b550-4a17-a44f-f64fe9773354 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:40.471209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3bbc2-b550-4a17-a44f-f64fe9773354 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:40.471489 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3bbc2-b550-4a17-a44f-f64fe9773354 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:40.472440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3bbc2-b550-4a17-a44f-f64fe9773354 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:40.474105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3bbc2-b550-4a17-a44f-f64fe9773354 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:40.505905 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 69/141] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:22:40 2022] POST /identity/v3/users => generated 284 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:22:41.254693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622ae2-e303-454a-b407-c3e24e844eaf None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:41.255257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622ae2-e303-454a-b407-c3e24e844eaf None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:41.255257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622ae2-e303-454a-b407-c3e24e844eaf None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:41.257864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 73/142] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:41.263435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d18fb-b476-4a0d-a800-5cb8e07926f4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:41.263940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d18fb-b476-4a0d-a800-5cb8e07926f4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:41.264044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d18fb-b476-4a0d-a800-5cb8e07926f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:41.306197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-635d18fb-b476-4a0d-a800-5cb8e07926f4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:41.308269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-635d18fb-b476-4a0d-a800-5cb8e07926f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:41.324979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86c624-5c21-407b-828e-4a375a087f74 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:41.325472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86c624-5c21-407b-828e-4a375a087f74 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:41.325964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86c624-5c21-407b-828e-4a375a087f74 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:41.328455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 74/143] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:41 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:41.337837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b543e-dc2a-4e88-ae74-65d4630775af None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:41.338350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b543e-dc2a-4e88-ae74-65d4630775af None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:41.338849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b543e-dc2a-4e88-ae74-65d4630775af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:41.368846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:41.383732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3f1b543e-dc2a-4e88-ae74-65d4630775af None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:41.391152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3f1b543e-dc2a-4e88-ae74-65d4630775af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:41.492304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 70/144] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:41.515022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:41.540039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 75/145] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:41.905153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d18fb-b476-4a0d-a800-5cb8e07926f4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:41.982521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b543e-dc2a-4e88-ae74-65d4630775af None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:42.084116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d18fb-b476-4a0d-a800-5cb8e07926f4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:42.145298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f560e50-2bc8-4ebd-b546-a9ea40a7c548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:42.146042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f560e50-2bc8-4ebd-b546-a9ea40a7c548 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:42.146388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f560e50-2bc8-4ebd-b546-a9ea40a7c548 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:42.146676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f560e50-2bc8-4ebd-b546-a9ea40a7c548 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:42.169620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f560e50-2bc8-4ebd-b546-a9ea40a7c548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4f560e50-2bc8-4ebd-b546-a9ea40a7c548 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:42.178690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:42.179597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:42.180806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 71/146] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:22:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:42.186084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f560e50-2bc8-4ebd-b546-a9ea40a7c548 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:42.211029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:42.223579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b543e-dc2a-4e88-ae74-65d4630775af None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:42.302309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97858c07-c50a-49bb-a743-d39f160c6fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:42.303836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97858c07-c50a-49bb-a743-d39f160c6fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:42.304334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97858c07-c50a-49bb-a743-d39f160c6fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:42.304822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97858c07-c50a-49bb-a743-d39f160c6fc4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:42.319463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97858c07-c50a-49bb-a743-d39f160c6fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-97858c07-c50a-49bb-a743-d39f160c6fc4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:42.324841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:42.330179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:22:42.330994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:42.332413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f560e50-2bc8-4ebd-b546-a9ea40a7c548 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:42.333898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 76/147] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:42.340505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97858c07-c50a-49bb-a743-d39f160c6fc4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:42.390576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91a999-d042-4f7e-b8fe-7ccd94dc83c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:42.391200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91a999-d042-4f7e-b8fe-7ccd94dc83c7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:42.391479 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91a999-d042-4f7e-b8fe-7ccd94dc83c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:42.391753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91a999-d042-4f7e-b8fe-7ccd94dc83c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:42.392529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91a999-d042-4f7e-b8fe-7ccd94dc83c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:42.392736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91a999-d042-4f7e-b8fe-7ccd94dc83c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:42.394198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91a999-d042-4f7e-b8fe-7ccd94dc83c7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:42.398088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:42.405241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 72/148] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:42 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:42.412649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91a999-d042-4f7e-b8fe-7ccd94dc83c7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:42.467434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97858c07-c50a-49bb-a743-d39f160c6fc4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:42.515674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608c42c-d140-4b17-88ae-dacde80afb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:42.516685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608c42c-d140-4b17-88ae-dacde80afb2b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:42.519806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608c42c-d140-4b17-88ae-dacde80afb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:42.520329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608c42c-d140-4b17-88ae-dacde80afb2b None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:42.521248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608c42c-d140-4b17-88ae-dacde80afb2b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:42.521817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608c42c-d140-4b17-88ae-dacde80afb2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:42.524208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608c42c-d140-4b17-88ae-dacde80afb2b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:42.533830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:42.544751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 77/149] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:42 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:42.550601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608c42c-d140-4b17-88ae-dacde80afb2b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:42.603591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91a999-d042-4f7e-b8fe-7ccd94dc83c7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:42.680849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abb5a6-b0d9-4b52-8cd3-3222100f5ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:42.681507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abb5a6-b0d9-4b52-8cd3-3222100f5ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:42.694619 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abb5a6-b0d9-4b52-8cd3-3222100f5ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:42.695138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abb5a6-b0d9-4b52-8cd3-3222100f5ec4 None admin] PATH_INFO: `/v3/users/d51a342066d64ae3b06abd4744cb8e8b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:42.696700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abb5a6-b0d9-4b52-8cd3-3222100f5ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=d51a342066d64ae3b06abd4744cb8e8b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:42.698559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abb5a6-b0d9-4b52-8cd3-3222100f5ec4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:42.700652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 73/150] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:42 2022] GET /identity/v3/users/d51a342066d64ae3b06abd4744cb8e8b => generated 248 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:42.701955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81a69e-8c14-482f-ba71-59e484b6e4e9 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:42.702240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81a69e-8c14-482f-ba71-59e484b6e4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:42.702511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81a69e-8c14-482f-ba71-59e484b6e4e9 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:42.703831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 74/151] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:42.714712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81a69e-8c14-482f-ba71-59e484b6e4e9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:42.720742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608c42c-d140-4b17-88ae-dacde80afb2b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:42.729215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:42.762614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e23c5-e763-4196-996b-3117872d4141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:42.763265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e23c5-e763-4196-996b-3117872d4141 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:42.763549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e23c5-e763-4196-996b-3117872d4141 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:42.763824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e23c5-e763-4196-996b-3117872d4141 None admin] PATH_INFO: `/v3/users/4fa10b3cc8464783913969ccc8c0c988` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:42.765102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e23c5-e763-4196-996b-3117872d4141 None admin] RBAC: Authorizing `identity:get_user(user_id=4fa10b3cc8464783913969ccc8c0c988)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:42.766673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e23c5-e763-4196-996b-3117872d4141 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:42.768392 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 78/152] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:42 2022] GET /identity/v3/users/4fa10b3cc8464783913969ccc8c0c988 => generated 290 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:42.769715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579cd4f-265d-413c-9179-3b5d9bd83f95 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:42.770005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579cd4f-265d-413c-9179-3b5d9bd83f95 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:42.770282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579cd4f-265d-413c-9179-3b5d9bd83f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:42.792143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c579cd4f-265d-413c-9179-3b5d9bd83f95 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:42.793440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c579cd4f-265d-413c-9179-3b5d9bd83f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:42.869127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 79/153] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:42.870179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579cd4f-265d-413c-9179-3b5d9bd83f95 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:42.908336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81a69e-8c14-482f-ba71-59e484b6e4e9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:42.962482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898409-d281-4c07-82f7-f1caab6b4fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:42.963104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898409-d281-4c07-82f7-f1caab6b4fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:42.963383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898409-d281-4c07-82f7-f1caab6b4fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:42.963657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898409-d281-4c07-82f7-f1caab6b4fc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:42.986020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:43.018048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:43.021494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579cd4f-265d-413c-9179-3b5d9bd83f95 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:43.086932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7af1af-67fa-4bd9-a57e-cad390c7a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:43.087592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7af1af-67fa-4bd9-a57e-cad390c7a928 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:43.087873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7af1af-67fa-4bd9-a57e-cad390c7a928 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:43.088329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7af1af-67fa-4bd9-a57e-cad390c7a928 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:43.094519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7af1af-67fa-4bd9-a57e-cad390c7a928 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:43.096049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7af1af-67fa-4bd9-a57e-cad390c7a928 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:43.096947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd898409-d281-4c07-82f7-f1caab6b4fc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:43.097862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 80/154] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:42 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:43.098806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd898409-d281-4c07-82f7-f1caab6b4fc8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:43.103853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7af1af-67fa-4bd9-a57e-cad390c7a928 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:43.300255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7af1af-67fa-4bd9-a57e-cad390c7a928 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:43.301749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 75/155] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:42 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:43.302728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898409-d281-4c07-82f7-f1caab6b4fc8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:43.339331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e684071-5720-4fd3-ab57-c40af50eb723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:43.340029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e684071-5720-4fd3-ab57-c40af50eb723 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:43.340366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e684071-5720-4fd3-ab57-c40af50eb723 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:43.340651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e684071-5720-4fd3-ab57-c40af50eb723 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1/users/4fa10b3cc8464783913969ccc8c0c988/roles/e272ecbda33d4b3e9308242d2d4ab302` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:43.342833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e684071-5720-4fd3-ab57-c40af50eb723 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc37cae8bab49beb9244a40222ffda1, user_id=4fa10b3cc8464783913969ccc8c0c988, role_id=e272ecbda33d4b3e9308242d2d4ab302)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:43.344396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e684071-5720-4fd3-ab57-c40af50eb723 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:43.361697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 81/156] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:22:43 2022] PUT /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1/users/4fa10b3cc8464783913969ccc8c0c988/roles/e272ecbda33d4b3e9308242d2d4ab302 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:22:43.376261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:43.423645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b8f0-dd42-44ee-8cf9-c72f743a0c19 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:43.424027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b8f0-dd42-44ee-8cf9-c72f743a0c19 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:43.424275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b8f0-dd42-44ee-8cf9-c72f743a0c19 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:43.425983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 82/157] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:43.434603 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a160f-ab2e-4405-9a22-1264e8699aa5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:43.434956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a160f-ab2e-4405-9a22-1264e8699aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:43.435273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a160f-ab2e-4405-9a22-1264e8699aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:43.459006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f19a160f-ab2e-4405-9a22-1264e8699aa5 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:43.461322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f19a160f-ab2e-4405-9a22-1264e8699aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:43.498292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898409-d281-4c07-82f7-f1caab6b4fc8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:43.499041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:43.552298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 83/158] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:43.567914 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae767ec-15cb-4ab0-898a-17f784ffdcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:43.568579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae767ec-15cb-4ab0-898a-17f784ffdcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:43.569856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae767ec-15cb-4ab0-898a-17f784ffdcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:43.570155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae767ec-15cb-4ab0-898a-17f784ffdcc7 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:43.574350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae767ec-15cb-4ab0-898a-17f784ffdcc7 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:43.575855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae767ec-15cb-4ab0-898a-17f784ffdcc7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:43.577534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 76/159] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:43 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:43.583683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a160f-ab2e-4405-9a22-1264e8699aa5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:43.595067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae767ec-15cb-4ab0-898a-17f784ffdcc7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:43.791624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae767ec-15cb-4ab0-898a-17f784ffdcc7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:43.835968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:43.852236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fda88a-35f0-4b61-80e4-97476e109a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:43.852886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fda88a-35f0-4b61-80e4-97476e109a01 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:43.853168 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fda88a-35f0-4b61-80e4-97476e109a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:43.853444 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fda88a-35f0-4b61-80e4-97476e109a01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:43.870870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fda88a-35f0-4b61-80e4-97476e109a01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a5fda88a-35f0-4b61-80e4-97476e109a01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:43.876317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:43.877233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:43.878094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 77/160] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:43.881742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a160f-ab2e-4405-9a22-1264e8699aa5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:43.881922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aae7a6-7667-458b-8f80-51bb0ea90a41 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:43.881922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aae7a6-7667-458b-8f80-51bb0ea90a41 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:43.881922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aae7a6-7667-458b-8f80-51bb0ea90a41 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:43.884527 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 78/161] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:43 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:43.887532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aae7a6-7667-458b-8f80-51bb0ea90a41 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:43.956665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2120e-5eef-428e-b7f6-198033159645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:43.957410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2120e-5eef-428e-b7f6-198033159645 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:43.957722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2120e-5eef-428e-b7f6-198033159645 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:43.957999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2120e-5eef-428e-b7f6-198033159645 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:43.958747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2120e-5eef-428e-b7f6-198033159645 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=6fc37cae8bab49beb9244a40222ffda1, user.id=d51a342066d64ae3b06abd4744cb8e8b) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:43.959054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2120e-5eef-428e-b7f6-198033159645 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:43.961182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2120e-5eef-428e-b7f6-198033159645 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:43.972188 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 84/162] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:43 2022] GET /identity/v3/role_assignments?user.id=d51a342066d64ae3b06abd4744cb8e8b&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 610 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:43.973471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c905ab-d86c-42dc-a74d-baeadcef0d11 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:43.973782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c905ab-d86c-42dc-a74d-baeadcef0d11 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:43.974057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c905ab-d86c-42dc-a74d-baeadcef0d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:44.014744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-30c905ab-d86c-42dc-a74d-baeadcef0d11 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:44.015972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-30c905ab-d86c-42dc-a74d-baeadcef0d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:44.053109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:44.082465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aae7a6-7667-458b-8f80-51bb0ea90a41 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:44.169031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 85/163] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:44.182707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d020a-9dde-4f30-a833-64d8846ff33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:44.183341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d020a-9dde-4f30-a833-64d8846ff33a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:44.183621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d020a-9dde-4f30-a833-64d8846ff33a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:44.183892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d020a-9dde-4f30-a833-64d8846ff33a None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:44.184578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d020a-9dde-4f30-a833-64d8846ff33a None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:44.184877 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d020a-9dde-4f30-a833-64d8846ff33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:44.186307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d020a-9dde-4f30-a833-64d8846ff33a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:44.198282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 79/164] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:43 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:44.204195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c905ab-d86c-42dc-a74d-baeadcef0d11 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:44.342764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c905ab-d86c-42dc-a74d-baeadcef0d11 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:44.384811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318dd39-ad4a-4775-ad26-0a6bb25e308a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:44.385469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318dd39-ad4a-4775-ad26-0a6bb25e308a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:44.385778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318dd39-ad4a-4775-ad26-0a6bb25e308a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:44.386054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318dd39-ad4a-4775-ad26-0a6bb25e308a None admin] PATH_INFO: `/v3/users/400981e92abc4ccea32372bcc765cfa8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:44.387262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318dd39-ad4a-4775-ad26-0a6bb25e308a None admin] RBAC: Authorizing `identity:get_user(user_id=400981e92abc4ccea32372bcc765cfa8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:44.388939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318dd39-ad4a-4775-ad26-0a6bb25e308a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:44.391653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 86/165] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:44 2022] GET /identity/v3/users/400981e92abc4ccea32372bcc765cfa8 => generated 276 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:44.396648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d020a-9dde-4f30-a833-64d8846ff33a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:44.404602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318dd39-ad4a-4775-ad26-0a6bb25e308a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:44.449684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:44.485961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:44.678850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d020a-9dde-4f30-a833-64d8846ff33a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:44.705309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318dd39-ad4a-4775-ad26-0a6bb25e308a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:44.752895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fbd2-1e45-46ea-9705-45c34587f17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:44.753972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fbd2-1e45-46ea-9705-45c34587f17f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:44.754460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fbd2-1e45-46ea-9705-45c34587f17f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:44.754917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fbd2-1e45-46ea-9705-45c34587f17f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:44.810365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5686a6a-5cfe-425f-932b-49a151211948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:44.811033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5686a6a-5cfe-425f-932b-49a151211948 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:44.811358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5686a6a-5cfe-425f-932b-49a151211948 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:44.811630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5686a6a-5cfe-425f-932b-49a151211948 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:44.842335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5686a6a-5cfe-425f-932b-49a151211948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e5686a6a-5cfe-425f-932b-49a151211948 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:44.847509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:44.848740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:44.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 87/166] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:22:44.858669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5686a6a-5cfe-425f-932b-49a151211948 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:44.900541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8fbd2-1e45-46ea-9705-45c34587f17f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:44.903454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8fbd2-1e45-46ea-9705-45c34587f17f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:44.966100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:45.092791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:45.138265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 80/167] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:44 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:45.139347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fbd2-1e45-46ea-9705-45c34587f17f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:45.151947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5686a6a-5cfe-425f-932b-49a151211948 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:45.214819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a201292-8a9d-4ebe-b728-942ab061afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:45.215445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201292-8a9d-4ebe-b728-942ab061afcd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:45.221637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201292-8a9d-4ebe-b728-942ab061afcd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:45.221914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201292-8a9d-4ebe-b728-942ab061afcd None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:45.222637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a201292-8a9d-4ebe-b728-942ab061afcd None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:45.222941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a201292-8a9d-4ebe-b728-942ab061afcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:45.224391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a201292-8a9d-4ebe-b728-942ab061afcd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:45.239833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 88/168] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:44 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:45.240529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a201292-8a9d-4ebe-b728-942ab061afcd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:45.411316 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fbd2-1e45-46ea-9705-45c34587f17f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:45.489450 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:45.500989 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a046c2-7b3d-4dc4-b5cd-bf76ea6d8528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:45.501659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a046c2-7b3d-4dc4-b5cd-bf76ea6d8528 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:45.501943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a046c2-7b3d-4dc4-b5cd-bf76ea6d8528 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:45.502189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a046c2-7b3d-4dc4-b5cd-bf76ea6d8528 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:45.530991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a046c2-7b3d-4dc4-b5cd-bf76ea6d8528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-47a046c2-7b3d-4dc4-b5cd-bf76ea6d8528 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:45.536180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:45.537089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:45.538660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:45.540086 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a201292-8a9d-4ebe-b728-942ab061afcd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:45.546268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 81/169] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:22:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:45.547226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a046c2-7b3d-4dc4-b5cd-bf76ea6d8528 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:45.628859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1218f89-1426-4bb4-9715-a8a082adcd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:45.629520 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1218f89-1426-4bb4-9715-a8a082adcd92 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:45.629836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1218f89-1426-4bb4-9715-a8a082adcd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:45.630117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1218f89-1426-4bb4-9715-a8a082adcd92 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:45.634374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1218f89-1426-4bb4-9715-a8a082adcd92 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:45.635888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1218f89-1426-4bb4-9715-a8a082adcd92 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:45.637824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 89/170] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:45 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:45.642111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ca40b-fa22-4f84-b59b-23335b65504f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:45.642404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ca40b-fa22-4f84-b59b-23335b65504f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:45.642688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ca40b-fa22-4f84-b59b-23335b65504f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:45.649967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 90/171] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:45 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:45.650940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ca40b-fa22-4f84-b59b-23335b65504f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:45.683794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:45.727382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a046c2-7b3d-4dc4-b5cd-bf76ea6d8528 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:45.809911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc33f67-6f4a-4607-b63e-3a221c7c760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:45.811318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc33f67-6f4a-4607-b63e-3a221c7c760b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:45.811931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc33f67-6f4a-4607-b63e-3a221c7c760b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:45.812586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc33f67-6f4a-4607-b63e-3a221c7c760b None admin] PATH_INFO: `/v3/users/400981e92abc4ccea32372bcc765cfa8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:45.816563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc33f67-6f4a-4607-b63e-3a221c7c760b None admin] RBAC: Authorizing `identity:get_user(user_id=400981e92abc4ccea32372bcc765cfa8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:45.819601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc33f67-6f4a-4607-b63e-3a221c7c760b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:45.829998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 82/172] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:45 2022] GET /identity/v3/users/400981e92abc4ccea32372bcc765cfa8 => generated 276 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:45.830848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc33f67-6f4a-4607-b63e-3a221c7c760b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:45.943111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ca40b-fa22-4f84-b59b-23335b65504f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:45.959231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:45.992966 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990b83b-a252-47af-8f16-dc267ea381d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:45.994361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990b83b-a252-47af-8f16-dc267ea381d2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:45.995178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990b83b-a252-47af-8f16-dc267ea381d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:45.995853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990b83b-a252-47af-8f16-dc267ea381d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:45.997297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990b83b-a252-47af-8f16-dc267ea381d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:45.998090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990b83b-a252-47af-8f16-dc267ea381d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:46.000985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990b83b-a252-47af-8f16-dc267ea381d2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:46.021960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 91/173] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:22:45 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:46.025541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980bc14-7bbb-4da1-850e-6726b90370e8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.026283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980bc14-7bbb-4da1-850e-6726b90370e8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.026924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980bc14-7bbb-4da1-850e-6726b90370e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.027064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc33f67-6f4a-4607-b63e-3a221c7c760b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:46.050031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c980bc14-7bbb-4da1-850e-6726b90370e8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:46.051236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c980bc14-7bbb-4da1-850e-6726b90370e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:46.078449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0d08-4c03-4dc8-8926-d4bafe95ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:46.079336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0d08-4c03-4dc8-8926-d4bafe95ce2f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.079805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0d08-4c03-4dc8-8926-d4bafe95ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.080273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0d08-4c03-4dc8-8926-d4bafe95ce2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.081237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a0d08-4c03-4dc8-8926-d4bafe95ce2f None admin] RBAC: Adding query filter params (role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=6fc37cae8bab49beb9244a40222ffda1, user.id=400981e92abc4ccea32372bcc765cfa8) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:46.081704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a0d08-4c03-4dc8-8926-d4bafe95ce2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:46.083560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a0d08-4c03-4dc8-8926-d4bafe95ce2f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:46.101651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 83/174] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:45 2022] GET /identity/v3/role_assignments?user.id=400981e92abc4ccea32372bcc765cfa8&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=90949fa670c2407ca92b16156e52f47f => generated 610 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:46.101651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0d08-4c03-4dc8-8926-d4bafe95ce2f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:46.163749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 92/175] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:46.166281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4c2d9-7fb6-4c0a-9629-191f61fde555 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.166877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4c2d9-7fb6-4c0a-9629-191f61fde555 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.167354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4c2d9-7fb6-4c0a-9629-191f61fde555 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.169110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 93/176] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:46 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:46.170668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4c2d9-7fb6-4c0a-9629-191f61fde555 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:46.174706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:46.315500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0d08-4c03-4dc8-8926-d4bafe95ce2f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:46.345118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:46.361911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4c2d9-7fb6-4c0a-9629-191f61fde555 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:46.363854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ed929-f81f-4948-bfe1-5b0dfdabed69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:46.364481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ed929-f81f-4948-bfe1-5b0dfdabed69 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.364772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ed929-f81f-4948-bfe1-5b0dfdabed69 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.365042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ed929-f81f-4948-bfe1-5b0dfdabed69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.406694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431df55d-8c79-4fe8-8dda-f2072026b651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:46.407342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431df55d-8c79-4fe8-8dda-f2072026b651 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.407621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431df55d-8c79-4fe8-8dda-f2072026b651 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.407892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431df55d-8c79-4fe8-8dda-f2072026b651 None admin] PATH_INFO: `/v3/users/400981e92abc4ccea32372bcc765cfa8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.409342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431df55d-8c79-4fe8-8dda-f2072026b651 None admin] RBAC: Authorizing `identity:get_user(user_id=400981e92abc4ccea32372bcc765cfa8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:46.410923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431df55d-8c79-4fe8-8dda-f2072026b651 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:46.413218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 94/177] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:46 2022] GET /identity/v3/users/400981e92abc4ccea32372bcc765cfa8 => generated 276 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:46.414595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c6f78-1cd2-40fb-bc48-10988bddcd5d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.414868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c6f78-1cd2-40fb-bc48-10988bddcd5d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.415307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c6f78-1cd2-40fb-bc48-10988bddcd5d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.416797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 95/178] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:46.418102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0877d3d-dd70-48dc-ae4b-b020c11344d7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.418528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0877d3d-dd70-48dc-ae4b-b020c11344d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.418987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0877d3d-dd70-48dc-ae4b-b020c11344d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.446384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a0877d3d-dd70-48dc-ae4b-b020c11344d7 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:46.447590 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a0877d3d-dd70-48dc-ae4b-b020c11344d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:46.454669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ed929-f81f-4948-bfe1-5b0dfdabed69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:46.456033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ed929-f81f-4948-bfe1-5b0dfdabed69 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:46.536977 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:46.543797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 96/179] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:46.544748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0877d3d-dd70-48dc-ae4b-b020c11344d7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:46.603784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 84/180] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:46.605339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b98e8b-8579-4a4d-9df9-2ac0e7c8bcc6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.605663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b98e8b-8579-4a4d-9df9-2ac0e7c8bcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.605945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b98e8b-8579-4a4d-9df9-2ac0e7c8bcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.626287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d6b98e8b-8579-4a4d-9df9-2ac0e7c8bcc6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:46.627647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d6b98e8b-8579-4a4d-9df9-2ac0e7c8bcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:46.645715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:46.695923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0877d3d-dd70-48dc-ae4b-b020c11344d7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:46.726462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 85/181] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:46.728107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35811310-d44c-4da7-9590-957deee23c98 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.728606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35811310-d44c-4da7-9590-957deee23c98 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.729044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35811310-d44c-4da7-9590-957deee23c98 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.730860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 86/182] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:46.731790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35811310-d44c-4da7-9590-957deee23c98 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:46.746340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916eda1-1300-42e0-9538-7ec55f6a1225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:46.746961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916eda1-1300-42e0-9538-7ec55f6a1225 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.747241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916eda1-1300-42e0-9538-7ec55f6a1225 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.747521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916eda1-1300-42e0-9538-7ec55f6a1225 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.835773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916eda1-1300-42e0-9538-7ec55f6a1225 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:46.837370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916eda1-1300-42e0-9538-7ec55f6a1225 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:46.840137 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:46.849691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35811310-d44c-4da7-9590-957deee23c98 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:46.889281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894a32-be20-4f2e-869a-296f4e45e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:46.890036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894a32-be20-4f2e-869a-296f4e45e66d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.890310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894a32-be20-4f2e-869a-296f4e45e66d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.890720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894a32-be20-4f2e-869a-296f4e45e66d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:46.904281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec894a32-be20-4f2e-869a-296f4e45e66d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-ec894a32-be20-4f2e-869a-296f4e45e66d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:46.909616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:46.911260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 87/183] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:46.912423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894a32-be20-4f2e-869a-296f4e45e66d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:46.927498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:46.973385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 97/184] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:46.975600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deba32-b3e8-415b-8c7b-4bc6215558b1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:46.975886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deba32-b3e8-415b-8c7b-4bc6215558b1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:46.976158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deba32-b3e8-415b-8c7b-4bc6215558b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.000076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d2deba32-b3e8-415b-8c7b-4bc6215558b1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:47.001273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d2deba32-b3e8-415b-8c7b-4bc6215558b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:47.038043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894a32-be20-4f2e-869a-296f4e45e66d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.085741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 98/185] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:47.085949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8342db5-8be7-4883-9ddf-48d037dd7ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.085949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8342db5-8be7-4883-9ddf-48d037dd7ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.086132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2deba32-b3e8-415b-8c7b-4bc6215558b1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.086212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8342db5-8be7-4883-9ddf-48d037dd7ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.086212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8342db5-8be7-4883-9ddf-48d037dd7ae6 None admin] PATH_INFO: `/v3/projects/af4040b3388e46e2b2c64306f872bbf4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.086997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8342db5-8be7-4883-9ddf-48d037dd7ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=af4040b3388e46e2b2c64306f872bbf4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:47.088507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8342db5-8be7-4883-9ddf-48d037dd7ae6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:47.090523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 88/186] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:46 2022] GET /identity/v3/projects/af4040b3388e46e2b2c64306f872bbf4 => generated 313 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:47.094644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8342db5-8be7-4883-9ddf-48d037dd7ae6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.125187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:47.203059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:47.218777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2deba32-b3e8-415b-8c7b-4bc6215558b1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.230613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8342db5-8be7-4883-9ddf-48d037dd7ae6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.267100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad2d03-7d02-476c-98ba-ca87b8de6ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.267733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad2d03-7d02-476c-98ba-ca87b8de6ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.268051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad2d03-7d02-476c-98ba-ca87b8de6ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.268329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad2d03-7d02-476c-98ba-ca87b8de6ff6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.269264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491edf41-09b6-464c-852b-5b4412eed8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.269908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491edf41-09b6-464c-852b-5b4412eed8fc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.270181 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491edf41-09b6-464c-852b-5b4412eed8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.270449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491edf41-09b6-464c-852b-5b4412eed8fc None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.271120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491edf41-09b6-464c-852b-5b4412eed8fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:47.271421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491edf41-09b6-464c-852b-5b4412eed8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:47.272943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491edf41-09b6-464c-852b-5b4412eed8fc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:47.282018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 89/187] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:47.282852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491edf41-09b6-464c-852b-5b4412eed8fc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.288101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad2d03-7d02-476c-98ba-ca87b8de6ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-54ad2d03-7d02-476c-98ba-ca87b8de6ff6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:47.293181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:47.294324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 99/188] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:47.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad2d03-7d02-476c-98ba-ca87b8de6ff6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.404537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491edf41-09b6-464c-852b-5b4412eed8fc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.404804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:47.417794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad2d03-7d02-476c-98ba-ca87b8de6ff6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.448206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47292e45-1a6e-4583-9cd6-24d8266ed7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.448835 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47292e45-1a6e-4583-9cd6-24d8266ed7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.449111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47292e45-1a6e-4583-9cd6-24d8266ed7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.449378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47292e45-1a6e-4583-9cd6-24d8266ed7b5 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.450653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47292e45-1a6e-4583-9cd6-24d8266ed7b5 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:47.452190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47292e45-1a6e-4583-9cd6-24d8266ed7b5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:47.453960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3120c1-032c-4805-aa1f-06315f19f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.454098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 90/189] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:47.454544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3120c1-032c-4805-aa1f-06315f19f483 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.454677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47292e45-1a6e-4583-9cd6-24d8266ed7b5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.454818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3120c1-032c-4805-aa1f-06315f19f483 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.455170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3120c1-032c-4805-aa1f-06315f19f483 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.456556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3120c1-032c-4805-aa1f-06315f19f483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:47.457988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3120c1-032c-4805-aa1f-06315f19f483 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:47.460360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 100/190] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/domains/default => generated 200 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:47.462545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3120c1-032c-4805-aa1f-06315f19f483 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.473720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:47.569638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47292e45-1a6e-4583-9cd6-24d8266ed7b5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.573894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3120c1-032c-4805-aa1f-06315f19f483 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.605942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20b79c-54ae-42ac-a692-26961c4c00d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.606562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20b79c-54ae-42ac-a692-26961c4c00d0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.606840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20b79c-54ae-42ac-a692-26961c4c00d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.607114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20b79c-54ae-42ac-a692-26961c4c00d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.607850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20b79c-54ae-42ac-a692-26961c4c00d0 None admin] RBAC: Adding query filter params (role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=af4040b3388e46e2b2c64306f872bbf4, user.id=400981e92abc4ccea32372bcc765cfa8) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:47.608162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20b79c-54ae-42ac-a692-26961c4c00d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:47.609644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20b79c-54ae-42ac-a692-26961c4c00d0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:47.612164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516489c-aebb-4996-8e0d-742e8aa21d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.612798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516489c-aebb-4996-8e0d-742e8aa21d10 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.613077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516489c-aebb-4996-8e0d-742e8aa21d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.613376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516489c-aebb-4996-8e0d-742e8aa21d10 None admin] PATH_INFO: `/v3/users/5c16680130a9444bb448a7f20de5ac52` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.616350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 91/191] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/role_assignments?user.id=400981e92abc4ccea32372bcc765cfa8&scope.project.id=af4040b3388e46e2b2c64306f872bbf4&role.id=90949fa670c2407ca92b16156e52f47f => generated 610 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:47.617103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20b79c-54ae-42ac-a692-26961c4c00d0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.629667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516489c-aebb-4996-8e0d-742e8aa21d10 None admin] RBAC: Authorizing `identity:get_user(user_id=5c16680130a9444bb448a7f20de5ac52)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:47.632109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516489c-aebb-4996-8e0d-742e8aa21d10 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:47.639443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 101/192] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/users/5c16680130a9444bb448a7f20de5ac52 => generated 284 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:47.640710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516489c-aebb-4996-8e0d-742e8aa21d10 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.667341 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:47.726952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:47.767734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20b79c-54ae-42ac-a692-26961c4c00d0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.796210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516489c-aebb-4996-8e0d-742e8aa21d10 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.814590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39714836-136a-4902-bc33-c0e089a593f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.815215 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39714836-136a-4902-bc33-c0e089a593f1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.815499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39714836-136a-4902-bc33-c0e089a593f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.815776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39714836-136a-4902-bc33-c0e089a593f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.817485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39714836-136a-4902-bc33-c0e089a593f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:47.817485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39714836-136a-4902-bc33-c0e089a593f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:47.819382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39714836-136a-4902-bc33-c0e089a593f1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:47.829894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 92/193] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:47.831097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39714836-136a-4902-bc33-c0e089a593f1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.850401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56b780-f6c1-4561-bb33-b8c813bca890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.851336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56b780-f6c1-4561-bb33-b8c813bca890 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.851622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56b780-f6c1-4561-bb33-b8c813bca890 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.851897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56b780-f6c1-4561-bb33-b8c813bca890 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.863621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56b780-f6c1-4561-bb33-b8c813bca890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-fc56b780-f6c1-4561-bb33-b8c813bca890 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:47.869434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:47.870364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:47.871141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 102/194] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:47.871799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56b780-f6c1-4561-bb33-b8c813bca890 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:47.940569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39714836-136a-4902-bc33-c0e089a593f1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.954153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:47.978468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9182fb5-2cd9-4b3f-adf5-53d626f6d5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:47.979005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9182fb5-2cd9-4b3f-adf5-53d626f6d5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:47.979284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9182fb5-2cd9-4b3f-adf5-53d626f6d5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:47.979456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56b780-f6c1-4561-bb33-b8c813bca890 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:47.979561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9182fb5-2cd9-4b3f-adf5-53d626f6d5a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:47.980284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9182fb5-2cd9-4b3f-adf5-53d626f6d5a5 None admin] RBAC: Adding query filter params (role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=6fc37cae8bab49beb9244a40222ffda1, user.id=400981e92abc4ccea32372bcc765cfa8) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:47.980588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9182fb5-2cd9-4b3f-adf5-53d626f6d5a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:47.982098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9182fb5-2cd9-4b3f-adf5-53d626f6d5a5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:47.984094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:47.993952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 93/195] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/role_assignments?user.id=400981e92abc4ccea32372bcc765cfa8&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=5dc7056ddc154b81939a17474a64f546 => generated 610 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:47.995024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9182fb5-2cd9-4b3f-adf5-53d626f6d5a5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:48.021149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21499b79-406f-40dc-bd7d-b6264c488d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:48.021827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21499b79-406f-40dc-bd7d-b6264c488d2f None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:48.022137 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21499b79-406f-40dc-bd7d-b6264c488d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:48.022553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21499b79-406f-40dc-bd7d-b6264c488d2f None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:48.023476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21499b79-406f-40dc-bd7d-b6264c488d2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:48.025474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21499b79-406f-40dc-bd7d-b6264c488d2f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:48.061629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 103/196] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:22:47 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:22:48.061629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21499b79-406f-40dc-bd7d-b6264c488d2f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:48.178372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9182fb5-2cd9-4b3f-adf5-53d626f6d5a5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:48.221509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21499b79-406f-40dc-bd7d-b6264c488d2f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:48.238010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16226a-3427-4e61-a75f-1f2b3b7b72c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:48.238557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16226a-3427-4e61-a75f-1f2b3b7b72c5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:48.238840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16226a-3427-4e61-a75f-1f2b3b7b72c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:48.239114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16226a-3427-4e61-a75f-1f2b3b7b72c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:48.263010 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:48.272749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d6254-5db9-4541-8711-b037904f11c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:48.273415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d6254-5db9-4541-8711-b037904f11c8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:48.273814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d6254-5db9-4541-8711-b037904f11c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:48.274089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d6254-5db9-4541-8711-b037904f11c8 None admin] PATH_INFO: `/v3/users/5c16680130a9444bb448a7f20de5ac52` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:48.276380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d6254-5db9-4541-8711-b037904f11c8 None admin] RBAC: Authorizing `identity:get_user(user_id=5c16680130a9444bb448a7f20de5ac52)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:48.278874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d6254-5db9-4541-8711-b037904f11c8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:48.280969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 104/197] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:48 2022] GET /identity/v3/users/5c16680130a9444bb448a7f20de5ac52 => generated 284 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:48.281716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d6254-5db9-4541-8711-b037904f11c8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:48.344209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:48.352461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16226a-3427-4e61-a75f-1f2b3b7b72c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:48.354225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16226a-3427-4e61-a75f-1f2b3b7b72c5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:48.438843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d6254-5db9-4541-8711-b037904f11c8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:48.497837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 94/198] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:48.498497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16226a-3427-4e61-a75f-1f2b3b7b72c5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:48.507135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89a1e7-8d96-4a09-b68c-00cbc2fdbe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:48.509416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89a1e7-8d96-4a09-b68c-00cbc2fdbe58 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:48.510224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89a1e7-8d96-4a09-b68c-00cbc2fdbe58 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:48.511533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89a1e7-8d96-4a09-b68c-00cbc2fdbe58 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:48.513931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89a1e7-8d96-4a09-b68c-00cbc2fdbe58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:48.514747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89a1e7-8d96-4a09-b68c-00cbc2fdbe58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:48.519055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89a1e7-8d96-4a09-b68c-00cbc2fdbe58 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:48.540627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 105/199] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:48 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:48.545691 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89a1e7-8d96-4a09-b68c-00cbc2fdbe58 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:48.676629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:48.701036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:48.710817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16226a-3427-4e61-a75f-1f2b3b7b72c5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:48.766538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89a1e7-8d96-4a09-b68c-00cbc2fdbe58 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:48.781299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0ca41-2906-459b-8508-47c1ee60e4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:48.782379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0ca41-2906-459b-8508-47c1ee60e4dc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:48.782379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0ca41-2906-459b-8508-47c1ee60e4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:48.782633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0ca41-2906-459b-8508-47c1ee60e4dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:48.842090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd2eb6-3762-4193-9eb1-b9e4304b34f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:48.843608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd2eb6-3762-4193-9eb1-b9e4304b34f1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:48.843892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd2eb6-3762-4193-9eb1-b9e4304b34f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:48.844177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd2eb6-3762-4193-9eb1-b9e4304b34f1 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:48.846278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd2eb6-3762-4193-9eb1-b9e4304b34f1 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:48.849196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd2eb6-3762-4193-9eb1-b9e4304b34f1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:48.853756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 106/200] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:48 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:48.854720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd2eb6-3762-4193-9eb1-b9e4304b34f1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:48.894815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0ca41-2906-459b-8508-47c1ee60e4dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:48.898864 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0ca41-2906-459b-8508-47c1ee60e4dc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:48.973831 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd2eb6-3762-4193-9eb1-b9e4304b34f1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:49.005973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:49.014872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b973c-9db7-4bfe-a656-4d546f4f65f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:49.015383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b973c-9db7-4bfe-a656-4d546f4f65f0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:49.015665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b973c-9db7-4bfe-a656-4d546f4f65f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:49.015941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b973c-9db7-4bfe-a656-4d546f4f65f0 None admin] PATH_INFO: `/v3/users/4fa10b3cc8464783913969ccc8c0c988` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:49.017410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b973c-9db7-4bfe-a656-4d546f4f65f0 None admin] RBAC: Authorizing `identity:get_user(user_id=4fa10b3cc8464783913969ccc8c0c988)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:49.019069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b973c-9db7-4bfe-a656-4d546f4f65f0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:49.020827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 107/201] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:48 2022] GET /identity/v3/users/4fa10b3cc8464783913969ccc8c0c988 => generated 290 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:49.021699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b973c-9db7-4bfe-a656-4d546f4f65f0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:49.119346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:49.135924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b973c-9db7-4bfe-a656-4d546f4f65f0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:49.136504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 95/202] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:48 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:49.138573 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0ca41-2906-459b-8508-47c1ee60e4dc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:49.174606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabc81c-c687-4dec-9c1f-0de4bf4ab33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:49.175227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabc81c-c687-4dec-9c1f-0de4bf4ab33d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:49.175506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabc81c-c687-4dec-9c1f-0de4bf4ab33d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:49.175779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabc81c-c687-4dec-9c1f-0de4bf4ab33d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:49.176556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabc81c-c687-4dec-9c1f-0de4bf4ab33d None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=4d0f6555111843d2af0acf5759071078, user.id=5c16680130a9444bb448a7f20de5ac52) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:49.176828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabc81c-c687-4dec-9c1f-0de4bf4ab33d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:49.178419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabc81c-c687-4dec-9c1f-0de4bf4ab33d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:49.185630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 108/203] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:49 2022] GET /identity/v3/role_assignments?user.id=5c16680130a9444bb448a7f20de5ac52&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:49.185957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabc81c-c687-4dec-9c1f-0de4bf4ab33d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:49.386103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:49.414152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0ca41-2906-459b-8508-47c1ee60e4dc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:49.471333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabc81c-c687-4dec-9c1f-0de4bf4ab33d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:49.476397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760ab1a-b6f1-41f1-bf13-f89e61f0f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:49.476858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760ab1a-b6f1-41f1-bf13-f89e61f0f0ca None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:49.477138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760ab1a-b6f1-41f1-bf13-f89e61f0f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:49.477406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760ab1a-b6f1-41f1-bf13-f89e61f0f0ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:49.553537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24aa92-e374-4dc0-8904-2d295d214889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:49.553537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24aa92-e374-4dc0-8904-2d295d214889 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:49.553537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24aa92-e374-4dc0-8904-2d295d214889 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:49.553537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24aa92-e374-4dc0-8904-2d295d214889 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:49.555427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24aa92-e374-4dc0-8904-2d295d214889 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:49.559381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24aa92-e374-4dc0-8904-2d295d214889 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:49.562271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 109/204] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:49 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:49.573226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24aa92-e374-4dc0-8904-2d295d214889 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:49.639832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:49.645977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760ab1a-b6f1-41f1-bf13-f89e61f0f0ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:49.652678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760ab1a-b6f1-41f1-bf13-f89e61f0f0ca None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:49.823037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24aa92-e374-4dc0-8904-2d295d214889 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:49.886012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:49.900813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 96/205] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:49 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:49.902604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076250fe-538e-4a4e-b312-6db78fe39caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:49.903523 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076250fe-538e-4a4e-b312-6db78fe39caf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:49.904061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076250fe-538e-4a4e-b312-6db78fe39caf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:49.904542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076250fe-538e-4a4e-b312-6db78fe39caf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:49.905504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076250fe-538e-4a4e-b312-6db78fe39caf None admin] RBAC: Adding query filter params (role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=4d0f6555111843d2af0acf5759071078, user.id=5c16680130a9444bb448a7f20de5ac52) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:49.906721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760ab1a-b6f1-41f1-bf13-f89e61f0f0ca None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:49.907290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076250fe-538e-4a4e-b312-6db78fe39caf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:49.910961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076250fe-538e-4a4e-b312-6db78fe39caf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:49.926273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 110/206] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:49 2022] GET /identity/v3/role_assignments?user.id=5c16680130a9444bb448a7f20de5ac52&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=5dc7056ddc154b81939a17474a64f546 => generated 260 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:50.037707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:50.043294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760ab1a-b6f1-41f1-bf13-f89e61f0f0ca None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:50.087179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fceee-e60d-4689-9970-754798482f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:50.088245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fceee-e60d-4689-9970-754798482f4c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:50.088827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fceee-e60d-4689-9970-754798482f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:50.089315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fceee-e60d-4689-9970-754798482f4c None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:50.091126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fceee-e60d-4689-9970-754798482f4c None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:50.092994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fceee-e60d-4689-9970-754798482f4c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:50.095172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 97/207] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:49 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:50.102627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076250fe-538e-4a4e-b312-6db78fe39caf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:50.284726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076250fe-538e-4a4e-b312-6db78fe39caf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:50.356324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cf917-833f-4e6b-8c77-e3c5f6f5268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:50.357304 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cf917-833f-4e6b-8c77-e3c5f6f5268b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:50.357837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cf917-833f-4e6b-8c77-e3c5f6f5268b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:50.358311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cf917-833f-4e6b-8c77-e3c5f6f5268b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:50.359258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cf917-833f-4e6b-8c77-e3c5f6f5268b None admin] RBAC: Adding query filter params (role.id=e272ecbda33d4b3e9308242d2d4ab302, scope.project.id=6fc37cae8bab49beb9244a40222ffda1, user.id=4fa10b3cc8464783913969ccc8c0c988) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:50.359778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cf917-833f-4e6b-8c77-e3c5f6f5268b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:50.361488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cf917-833f-4e6b-8c77-e3c5f6f5268b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:50.375011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 111/208] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:50 2022] GET /identity/v3/role_assignments?user.id=4fa10b3cc8464783913969ccc8c0c988&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=e272ecbda33d4b3e9308242d2d4ab302 => generated 610 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:50.848706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3259d2d-79d3-49cd-95df-68cee9c8ef5e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:50.849810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3259d2d-79d3-49cd-95df-68cee9c8ef5e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:50.850695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3259d2d-79d3-49cd-95df-68cee9c8ef5e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:50.852699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 98/209] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:50 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:50.858954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654fe22-9ad3-46ca-bccd-fcd649f63c20 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:50.859919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654fe22-9ad3-46ca-bccd-fcd649f63c20 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:50.860395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654fe22-9ad3-46ca-bccd-fcd649f63c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:50.882730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7654fe22-9ad3-46ca-bccd-fcd649f63c20 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:50.884574 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7654fe22-9ad3-46ca-bccd-fcd649f63c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:50.959438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:50.979245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 112/210] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:51.123884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719f45a-cf46-44b3-b82e-d695ca700fe5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:51.124520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719f45a-cf46-44b3-b82e-d695ca700fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:51.125002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719f45a-cf46-44b3-b82e-d695ca700fe5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:51.126984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 99/211] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:51.136288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8aef9-0917-4721-b216-80e07b87b7e3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:51.136684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8aef9-0917-4721-b216-80e07b87b7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:51.137071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8aef9-0917-4721-b216-80e07b87b7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:51.174405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-afd8aef9-0917-4721-b216-80e07b87b7e3 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:51.176482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-afd8aef9-0917-4721-b216-80e07b87b7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:51.302820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 113/212] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:51.389668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719f45a-cf46-44b3-b82e-d695ca700fe5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:51.583023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8aef9-0917-4721-b216-80e07b87b7e3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:51.589715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719f45a-cf46-44b3-b82e-d695ca700fe5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:51.643340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e8071-8160-4dcd-9213-654a25a69c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:51.644003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e8071-8160-4dcd-9213-654a25a69c15 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:51.644281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e8071-8160-4dcd-9213-654a25a69c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:51.644551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e8071-8160-4dcd-9213-654a25a69c15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:51.645825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e8071-8160-4dcd-9213-654a25a69c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:51.648138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e8071-8160-4dcd-9213-654a25a69c15 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:51.650479 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 100/213] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:51 2022] GET /identity/v3/domains/default => generated 200 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:51.656111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e8071-8160-4dcd-9213-654a25a69c15 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:51.698680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:51.758032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8aef9-0917-4721-b216-80e07b87b7e3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:51.784717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:51.808207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e8071-8160-4dcd-9213-654a25a69c15 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:51.808549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757214-6ff7-476c-bdcc-c12aef65f5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:51.809203 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757214-6ff7-476c-bdcc-c12aef65f5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:51.809511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757214-6ff7-476c-bdcc-c12aef65f5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:51.809884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757214-6ff7-476c-bdcc-c12aef65f5c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:51.828254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757214-6ff7-476c-bdcc-c12aef65f5c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d2757214-6ff7-476c-bdcc-c12aef65f5c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:51.835715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:51.836712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:51.837531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 114/214] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:51.845059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757214-6ff7-476c-bdcc-c12aef65f5c0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:51.856351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230d83b-05b0-458c-89e7-fbe57f9071bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:51.857038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230d83b-05b0-458c-89e7-fbe57f9071bf None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:51.857246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230d83b-05b0-458c-89e7-fbe57f9071bf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:51.857520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230d83b-05b0-458c-89e7-fbe57f9071bf None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:51.858228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230d83b-05b0-458c-89e7-fbe57f9071bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:51.859704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230d83b-05b0-458c-89e7-fbe57f9071bf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:51.895251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 101/215] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:22:51 2022] POST /identity/v3/users => generated 298 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:22:51.938399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701b5ce-6627-4822-9409-10c619aef752 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:51.939036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701b5ce-6627-4822-9409-10c619aef752 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:51.939516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701b5ce-6627-4822-9409-10c619aef752 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:51.942544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 102/216] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:51 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:22:51.948398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d6e95-deb8-4541-9ee7-ff3d6a4404e6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:51.948734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d6e95-deb8-4541-9ee7-ff3d6a4404e6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:51.948963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d6e95-deb8-4541-9ee7-ff3d6a4404e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:51.976571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5c1d6e95-deb8-4541-9ee7-ff3d6a4404e6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:51.977890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5c1d6e95-deb8-4541-9ee7-ff3d6a4404e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:52.012881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757214-6ff7-476c-bdcc-c12aef65f5c0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:52.056834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb057-21ea-4b08-8c34-3b09a4b93115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:52.057878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb057-21ea-4b08-8c34-3b09a4b93115 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:52.058368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb057-21ea-4b08-8c34-3b09a4b93115 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:52.058867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb057-21ea-4b08-8c34-3b09a4b93115 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:52.059759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fb057-21ea-4b08-8c34-3b09a4b93115 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:52.060059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fb057-21ea-4b08-8c34-3b09a4b93115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:52.061496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fb057-21ea-4b08-8c34-3b09a4b93115 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:52.065050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 103/217] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:52.072378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 115/218] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:51 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:52.078492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d6e95-deb8-4541-9ee7-ff3d6a4404e6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:52.105358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:52.234131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d6e95-deb8-4541-9ee7-ff3d6a4404e6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:52.299442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e52fec-c004-4af0-8fb6-7b3ca7d73d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:52.300106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e52fec-c004-4af0-8fb6-7b3ca7d73d3b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:52.300395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e52fec-c004-4af0-8fb6-7b3ca7d73d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:52.300669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e52fec-c004-4af0-8fb6-7b3ca7d73d3b None admin] PATH_INFO: `/v3/users/981f5558161b450b8327c26a0cdabc8e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:52.323612 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e52fec-c004-4af0-8fb6-7b3ca7d73d3b None admin] RBAC: Authorizing `identity:get_user(user_id=981f5558161b450b8327c26a0cdabc8e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:52.326897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e52fec-c004-4af0-8fb6-7b3ca7d73d3b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:52.330313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 104/219] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:52 2022] GET /identity/v3/users/981f5558161b450b8327c26a0cdabc8e => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:52.340739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb057-21ea-4b08-8c34-3b09a4b93115 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:52.418700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:52.446358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e52fec-c004-4af0-8fb6-7b3ca7d73d3b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:52.548871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb057-21ea-4b08-8c34-3b09a4b93115 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:52.602026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:52.635201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddae70-f7e1-47e3-9169-62dd6df97254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:52.635829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddae70-f7e1-47e3-9169-62dd6df97254 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:52.636109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddae70-f7e1-47e3-9169-62dd6df97254 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:52.636378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddae70-f7e1-47e3-9169-62dd6df97254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:52.654187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e52fec-c004-4af0-8fb6-7b3ca7d73d3b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:52.738512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a37419-8a00-4187-ae4e-3960a58bef1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:52.739088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a37419-8a00-4187-ae4e-3960a58bef1b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:52.739363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a37419-8a00-4187-ae4e-3960a58bef1b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:52.739643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a37419-8a00-4187-ae4e-3960a58bef1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:52.764944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a37419-8a00-4187-ae4e-3960a58bef1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-64a37419-8a00-4187-ae4e-3960a58bef1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:52.770150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:52.771124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:52.772541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:52.773564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 105/220] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:22:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:52.786694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a37419-8a00-4187-ae4e-3960a58bef1b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:52.794361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddae70-f7e1-47e3-9169-62dd6df97254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:52.796353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddae70-f7e1-47e3-9169-62dd6df97254 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:52.862136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:52.928469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 116/221] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:52 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:52.935683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b379f28-b572-40c4-abe8-c7a2c63140fa None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:52.936277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b379f28-b572-40c4-abe8-c7a2c63140fa None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:52.936938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b379f28-b572-40c4-abe8-c7a2c63140fa None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:52.938680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 117/222] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:52.939742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b379f28-b572-40c4-abe8-c7a2c63140fa None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:52.985890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a37419-8a00-4187-ae4e-3960a58bef1b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:53.029401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf8053-a1ce-4c01-a216-605584989a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:53.030078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8053-a1ce-4c01-a216-605584989a3b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:53.030460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8053-a1ce-4c01-a216-605584989a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:53.030836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8053-a1ce-4c01-a216-605584989a3b None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:53.031665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf8053-a1ce-4c01-a216-605584989a3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:53.032070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf8053-a1ce-4c01-a216-605584989a3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:53.033768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf8053-a1ce-4c01-a216-605584989a3b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:53.040599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:53.045164 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 106/223] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:52 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:53.047056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b7f9-b937-47cf-93fb-a62351470dea None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:53.047231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b7f9-b937-47cf-93fb-a62351470dea None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:53.047865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b7f9-b937-47cf-93fb-a62351470dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:53.069163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-38e0b7f9-b937-47cf-93fb-a62351470dea None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:53.070886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-38e0b7f9-b937-47cf-93fb-a62351470dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:53.078442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b379f28-b572-40c4-abe8-c7a2c63140fa None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:53.120794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a92bf-278b-4192-a8a3-fa159de4810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:53.121860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a92bf-278b-4192-a8a3-fa159de4810a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:53.122356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a92bf-278b-4192-a8a3-fa159de4810a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:53.122830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a92bf-278b-4192-a8a3-fa159de4810a None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:53.124294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a92bf-278b-4192-a8a3-fa159de4810a None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:53.126282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a92bf-278b-4192-a8a3-fa159de4810a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:53.128342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 118/224] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:52 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:53.134999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a92bf-278b-4192-a8a3-fa159de4810a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:53.148411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:53.165849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 107/225] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:53.167176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b7f9-b937-47cf-93fb-a62351470dea None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:53.361760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a92bf-278b-4192-a8a3-fa159de4810a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:53.423320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b7f9-b937-47cf-93fb-a62351470dea None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:53.437979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f7f78-1bd2-445e-97c3-b04d75b9e819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:53.439027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f7f78-1bd2-445e-97c3-b04d75b9e819 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:53.439464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f7f78-1bd2-445e-97c3-b04d75b9e819 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:53.439912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f7f78-1bd2-445e-97c3-b04d75b9e819 None admin] PATH_INFO: `/v3/users/5c16680130a9444bb448a7f20de5ac52` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:53.441363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f7f78-1bd2-445e-97c3-b04d75b9e819 None admin] RBAC: Authorizing `identity:get_user(user_id=5c16680130a9444bb448a7f20de5ac52)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:53.448325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f7f78-1bd2-445e-97c3-b04d75b9e819 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:53.450480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 119/226] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:53 2022] GET /identity/v3/users/5c16680130a9444bb448a7f20de5ac52 => generated 284 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:53.458065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f7f78-1bd2-445e-97c3-b04d75b9e819 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:53.464656 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:53.479050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d9911-a226-48aa-a171-ec249e3ba328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:53.480290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9911-a226-48aa-a171-ec249e3ba328 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:53.480728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9911-a226-48aa-a171-ec249e3ba328 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:53.481377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9911-a226-48aa-a171-ec249e3ba328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:53.482501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d9911-a226-48aa-a171-ec249e3ba328 None admin] RBAC: Adding query filter params (role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=4d0f6555111843d2af0acf5759071078, user.id=981f5558161b450b8327c26a0cdabc8e) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:53.483263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d9911-a226-48aa-a171-ec249e3ba328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:53.485362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d9911-a226-48aa-a171-ec249e3ba328 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:53.508421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 108/227] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:53 2022] GET /identity/v3/role_assignments?user.id=981f5558161b450b8327c26a0cdabc8e&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=90949fa670c2407ca92b16156e52f47f => generated 260 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:53.509785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d9911-a226-48aa-a171-ec249e3ba328 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:53.595764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:53.628799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f7f78-1bd2-445e-97c3-b04d75b9e819 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:53.676191 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae64e7-d873-45ae-bb84-183e256d8bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:53.676852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae64e7-d873-45ae-bb84-183e256d8bda None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:53.677132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae64e7-d873-45ae-bb84-183e256d8bda None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:53.677403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae64e7-d873-45ae-bb84-183e256d8bda None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:53.679678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae64e7-d873-45ae-bb84-183e256d8bda None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:53.681685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae64e7-d873-45ae-bb84-183e256d8bda None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:53.684557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 120/228] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:53 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:53.692044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae64e7-d873-45ae-bb84-183e256d8bda None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:53.823585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d9911-a226-48aa-a171-ec249e3ba328 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:53.891964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae64e7-d873-45ae-bb84-183e256d8bda None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:53.943043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed339aae-98f4-42f0-a002-047f45d9ff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:53.943738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed339aae-98f4-42f0-a002-047f45d9ff0e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:53.944666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed339aae-98f4-42f0-a002-047f45d9ff0e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:53.944666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed339aae-98f4-42f0-a002-047f45d9ff0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:53.966024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31532d-16cc-4cad-bbd9-f4623afeab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:53.967147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31532d-16cc-4cad-bbd9-f4623afeab70 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:53.967919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31532d-16cc-4cad-bbd9-f4623afeab70 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:53.968377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31532d-16cc-4cad-bbd9-f4623afeab70 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078/users/5c16680130a9444bb448a7f20de5ac52/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:53.972578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31532d-16cc-4cad-bbd9-f4623afeab70 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d0f6555111843d2af0acf5759071078, user_id=5c16680130a9444bb448a7f20de5ac52, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:53.976340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31532d-16cc-4cad-bbd9-f4623afeab70 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:53.982315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed339aae-98f4-42f0-a002-047f45d9ff0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-ed339aae-98f4-42f0-a002-047f45d9ff0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:53.989780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:53.993176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:53.998961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 109/229] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:22:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:54.008546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 121/230] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:22:53 2022] PUT /identity/v3/projects/4d0f6555111843d2af0acf5759071078/users/5c16680130a9444bb448a7f20de5ac52/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:22:54.008989 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed339aae-98f4-42f0-a002-047f45d9ff0e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:54.154410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:54.248744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed339aae-98f4-42f0-a002-047f45d9ff0e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:54.299155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc46234-6dd1-40b1-a208-4bd8ca03933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:54.299894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46234-6dd1-40b1-a208-4bd8ca03933d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:54.300236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46234-6dd1-40b1-a208-4bd8ca03933d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:54.300578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46234-6dd1-40b1-a208-4bd8ca03933d None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:54.301415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc46234-6dd1-40b1-a208-4bd8ca03933d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:54.301967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc46234-6dd1-40b1-a208-4bd8ca03933d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:54.304365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc46234-6dd1-40b1-a208-4bd8ca03933d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:54.314053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 110/231] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:22:54 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:54.321366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31532d-16cc-4cad-bbd9-f4623afeab70 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:54.327471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:54.550884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31532d-16cc-4cad-bbd9-f4623afeab70 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:54.617875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8608f25-2ebd-47e8-8921-d930442e1227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:54.618828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8608f25-2ebd-47e8-8921-d930442e1227 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:54.619282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8608f25-2ebd-47e8-8921-d930442e1227 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:54.619718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8608f25-2ebd-47e8-8921-d930442e1227 None admin] PATH_INFO: `/v3/users/5c16680130a9444bb448a7f20de5ac52` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:54.621466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8608f25-2ebd-47e8-8921-d930442e1227 None admin] RBAC: Authorizing `identity:get_user(user_id=5c16680130a9444bb448a7f20de5ac52)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:54.624104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8608f25-2ebd-47e8-8921-d930442e1227 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:54.627613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 122/232] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:54 2022] GET /identity/v3/users/5c16680130a9444bb448a7f20de5ac52 => generated 284 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:54.634152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc46234-6dd1-40b1-a208-4bd8ca03933d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:54.840566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc46234-6dd1-40b1-a208-4bd8ca03933d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:54.869416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:54.891009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1627fe5-0680-456e-a18a-dcf30dc9f185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:54.891657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1627fe5-0680-456e-a18a-dcf30dc9f185 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:54.891932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1627fe5-0680-456e-a18a-dcf30dc9f185 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:54.892200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1627fe5-0680-456e-a18a-dcf30dc9f185 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:54.893401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1627fe5-0680-456e-a18a-dcf30dc9f185 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:54.894902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1627fe5-0680-456e-a18a-dcf30dc9f185 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:54.897511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 111/233] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:54 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:54.904959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8608f25-2ebd-47e8-8921-d930442e1227 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:54.949819 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8315d-e6ac-465d-909f-2d979256489e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:54.950382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8315d-e6ac-465d-909f-2d979256489e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:54.950658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8315d-e6ac-465d-909f-2d979256489e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:54.952424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 112/234] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:22:54.958404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f205efd-0c0c-4c51-ac14-f92b961e9f05 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:54.958721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f205efd-0c0c-4c51-ac14-f92b961e9f05 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:54.958952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f205efd-0c0c-4c51-ac14-f92b961e9f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:54.984085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3f205efd-0c0c-4c51-ac14-f92b961e9f05 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:54.985376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3f205efd-0c0c-4c51-ac14-f92b961e9f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:55.072378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:55.093857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 113/235] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:55.096467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fab27c-55b2-483b-a764-a4ffbdf5c15c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:55.097050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fab27c-55b2-483b-a764-a4ffbdf5c15c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:55.097646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fab27c-55b2-483b-a764-a4ffbdf5c15c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:55.099167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8608f25-2ebd-47e8-8921-d930442e1227 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:55.100326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 114/236] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:55 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:55.112342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57563b-4d6b-4c0d-935a-83d3c5956942 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:55.112835 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57563b-4d6b-4c0d-935a-83d3c5956942 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:55.113322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57563b-4d6b-4c0d-935a-83d3c5956942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:55.155225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989342c9-cefd-4db9-bb79-46ee2fae1155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:55.156489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989342c9-cefd-4db9-bb79-46ee2fae1155 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:55.156974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989342c9-cefd-4db9-bb79-46ee2fae1155 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:55.159442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989342c9-cefd-4db9-bb79-46ee2fae1155 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078/users/5c16680130a9444bb448a7f20de5ac52/roles/5dc7056ddc154b81939a17474a64f546` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:55.167549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989342c9-cefd-4db9-bb79-46ee2fae1155 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d0f6555111843d2af0acf5759071078, user_id=5c16680130a9444bb448a7f20de5ac52, role_id=5dc7056ddc154b81939a17474a64f546)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:55.175329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989342c9-cefd-4db9-bb79-46ee2fae1155 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:55.177441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cc57563b-4d6b-4c0d-935a-83d3c5956942 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:55.179683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cc57563b-4d6b-4c0d-935a-83d3c5956942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:55.219916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 123/237] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:22:54 2022] PUT /identity/v3/projects/4d0f6555111843d2af0acf5759071078/users/5c16680130a9444bb448a7f20de5ac52/roles/5dc7056ddc154b81939a17474a64f546 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:22:55.350181 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 115/238] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:55.535486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989342c9-cefd-4db9-bb79-46ee2fae1155 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:55.689191 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989342c9-cefd-4db9-bb79-46ee2fae1155 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:55.755452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:55.762245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39385034-2ec0-4e0e-a2ac-73e56dfe301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:55.763449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39385034-2ec0-4e0e-a2ac-73e56dfe301a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:55.763766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39385034-2ec0-4e0e-a2ac-73e56dfe301a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:55.764064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39385034-2ec0-4e0e-a2ac-73e56dfe301a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:55.785904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39385034-2ec0-4e0e-a2ac-73e56dfe301a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-39385034-2ec0-4e0e-a2ac-73e56dfe301a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:55.792182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:55.793118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:55.793946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 124/239] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:55.804347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57563b-4d6b-4c0d-935a-83d3c5956942 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:55.896741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:55.980446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57563b-4d6b-4c0d-935a-83d3c5956942 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:56.003564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39385034-2ec0-4e0e-a2ac-73e56dfe301a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:56.029091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe17b40-9d42-4afe-b7dc-3448514ff8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:56.029736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe17b40-9d42-4afe-b7dc-3448514ff8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.030013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe17b40-9d42-4afe-b7dc-3448514ff8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.030286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe17b40-9d42-4afe-b7dc-3448514ff8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:56.030944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe17b40-9d42-4afe-b7dc-3448514ff8a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:56.031240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe17b40-9d42-4afe-b7dc-3448514ff8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:56.032632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe17b40-9d42-4afe-b7dc-3448514ff8a3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:56.056823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 116/240] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:55 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:56.066253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe17b40-9d42-4afe-b7dc-3448514ff8a3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:56.140374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39385034-2ec0-4e0e-a2ac-73e56dfe301a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:56.189207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e932bc2-f3ea-4a48-a3a9-950f5e1c914c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:56.190328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e932bc2-f3ea-4a48-a3a9-950f5e1c914c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.190825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e932bc2-f3ea-4a48-a3a9-950f5e1c914c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.191318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e932bc2-f3ea-4a48-a3a9-950f5e1c914c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:56.197940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e932bc2-f3ea-4a48-a3a9-950f5e1c914c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:56.200744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e932bc2-f3ea-4a48-a3a9-950f5e1c914c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:56.210284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 125/241] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:22:56 2022] GET /identity/v3/domains/default => generated 200 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:56.217110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e932bc2-f3ea-4a48-a3a9-950f5e1c914c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:56.311156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe17b40-9d42-4afe-b7dc-3448514ff8a3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:56.319681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:56.355767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055849b-9cfb-41a4-a3c8-07390fa289ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:56.356668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055849b-9cfb-41a4-a3c8-07390fa289ff None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.357096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055849b-9cfb-41a4-a3c8-07390fa289ff None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.357543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055849b-9cfb-41a4-a3c8-07390fa289ff None admin] PATH_INFO: `/v3/users/01a02236d2cf4ec0beab6f4214b37940` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:56.374204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055849b-9cfb-41a4-a3c8-07390fa289ff None admin] RBAC: Authorizing `identity:get_user(user_id=01a02236d2cf4ec0beab6f4214b37940)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:56.375961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055849b-9cfb-41a4-a3c8-07390fa289ff None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:56.377913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 117/242] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:56 2022] GET /identity/v3/users/01a02236d2cf4ec0beab6f4214b37940 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:56.384528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055849b-9cfb-41a4-a3c8-07390fa289ff None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:56.419704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:56.507062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055849b-9cfb-41a4-a3c8-07390fa289ff None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:56.518190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e932bc2-f3ea-4a48-a3a9-950f5e1c914c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:56.564482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4f2af-b242-4509-b679-03c78a7300a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:56.565563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4f2af-b242-4509-b679-03c78a7300a7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.566138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4f2af-b242-4509-b679-03c78a7300a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.566566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4f2af-b242-4509-b679-03c78a7300a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:56.568035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85087038-969d-49cb-8427-d9b35138bbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:56.568819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85087038-969d-49cb-8427-d9b35138bbf1 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.569348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85087038-969d-49cb-8427-d9b35138bbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.570005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85087038-969d-49cb-8427-d9b35138bbf1 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:56.570784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85087038-969d-49cb-8427-d9b35138bbf1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:56.572613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85087038-969d-49cb-8427-d9b35138bbf1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:56.606245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 126/243] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:22:56 2022] POST /identity/v3/users => generated 294 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:22:56.606245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053798f1-bb05-4902-8b0a-a9f3a59b653b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.606245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053798f1-bb05-4902-8b0a-a9f3a59b653b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.606245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053798f1-bb05-4902-8b0a-a9f3a59b653b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:56.609762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 127/244] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:56 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:56.613632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fa52f-d85f-4af9-86cb-7910850d6c58 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.613632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fa52f-d85f-4af9-86cb-7910850d6c58 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.613632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fa52f-d85f-4af9-86cb-7910850d6c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:56.630563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:56.639436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1b5fa52f-d85f-4af9-86cb-7910850d6c58 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:56.640732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1b5fa52f-d85f-4af9-86cb-7910850d6c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:56.745602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 128/245] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:56.774952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4f2af-b242-4509-b679-03c78a7300a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:56.776746 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4f2af-b242-4509-b679-03c78a7300a7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:56.940286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 118/246] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:56 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:22:56.947726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fa52f-d85f-4af9-86cb-7910850d6c58 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:56.970873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e777035-9663-4fcb-8364-f4ddea2a8934 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.971622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e777035-9663-4fcb-8364-f4ddea2a8934 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.972642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e777035-9663-4fcb-8364-f4ddea2a8934 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:56.975707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 119/247] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:56 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:56.981269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a8e73-31e7-470a-b55c-f566df4cc043 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:56.981804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a8e73-31e7-470a-b55c-f566df4cc043 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:56.982223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a8e73-31e7-470a-b55c-f566df4cc043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:57.007291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cf3a8e73-31e7-470a-b55c-f566df4cc043 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:57.008801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cf3a8e73-31e7-470a-b55c-f566df4cc043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:57.041017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:57.082761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:57.094221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 120/248] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:22:57.095780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a8e73-31e7-470a-b55c-f566df4cc043 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:57.123269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fa52f-d85f-4af9-86cb-7910850d6c58 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:57.162109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470dba2-4098-4fd0-a0e5-1615756745e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:57.163857 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470dba2-4098-4fd0-a0e5-1615756745e1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:57.164345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470dba2-4098-4fd0-a0e5-1615756745e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:57.164838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470dba2-4098-4fd0-a0e5-1615756745e1 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:57.166331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470dba2-4098-4fd0-a0e5-1615756745e1 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:57.168082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470dba2-4098-4fd0-a0e5-1615756745e1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:57.169972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 129/249] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:56 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:57.176065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470dba2-4098-4fd0-a0e5-1615756745e1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:57.213015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a8e73-31e7-470a-b55c-f566df4cc043 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:57.251752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c037db-c112-45f4-966f-cacdfe3e1e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:57.252627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c037db-c112-45f4-966f-cacdfe3e1e38 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:57.253067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c037db-c112-45f4-966f-cacdfe3e1e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:57.253472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c037db-c112-45f4-966f-cacdfe3e1e38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:57.265885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c037db-c112-45f4-966f-cacdfe3e1e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-73c037db-c112-45f4-966f-cacdfe3e1e38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:57.271094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:57.272872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:22:57.274570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:22:57.277251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 121/250] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:57.283147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c037db-c112-45f4-966f-cacdfe3e1e38 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:57.291623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470dba2-4098-4fd0-a0e5-1615756745e1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:57.324009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:57.338273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c4a04-3116-4873-9194-3f16ba93a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:57.339952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c4a04-3116-4873-9194-3f16ba93a233 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:57.340612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c4a04-3116-4873-9194-3f16ba93a233 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:57.341012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c4a04-3116-4873-9194-3f16ba93a233 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:57.341952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c4a04-3116-4873-9194-3f16ba93a233 None admin] RBAC: Adding query filter params (role.id=e272ecbda33d4b3e9308242d2d4ab302, scope.project.id=4d0f6555111843d2af0acf5759071078, user.id=01a02236d2cf4ec0beab6f4214b37940) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:57.342403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c4a04-3116-4873-9194-3f16ba93a233 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:57.343995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c4a04-3116-4873-9194-3f16ba93a233 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:57.350186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 130/251] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:57 2022] GET /identity/v3/role_assignments?user.id=01a02236d2cf4ec0beab6f4214b37940&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=e272ecbda33d4b3e9308242d2d4ab302 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:57.370999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:57.415109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c037db-c112-45f4-966f-cacdfe3e1e38 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:57.469282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c4a04-3116-4873-9194-3f16ba93a233 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:57.469669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafec27-c177-49e7-9c1e-6a5ac1e4a7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:57.470315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafec27-c177-49e7-9c1e-6a5ac1e4a7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:57.470654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafec27-c177-49e7-9c1e-6a5ac1e4a7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:57.470928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafec27-c177-49e7-9c1e-6a5ac1e4a7a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:57.471652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafec27-c177-49e7-9c1e-6a5ac1e4a7a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:57.471957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafec27-c177-49e7-9c1e-6a5ac1e4a7a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:57.473371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafec27-c177-49e7-9c1e-6a5ac1e4a7a3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:57.484300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 122/252] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:22:57 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:57.490371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafec27-c177-49e7-9c1e-6a5ac1e4a7a3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:57.712604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c4a04-3116-4873-9194-3f16ba93a233 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:57.718706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafec27-c177-49e7-9c1e-6a5ac1e4a7a3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:57.745834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:57.756667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d9979-09df-4398-acde-3298412ad802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:57.756667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d9979-09df-4398-acde-3298412ad802 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:57.756667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d9979-09df-4398-acde-3298412ad802 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:57.756667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d9979-09df-4398-acde-3298412ad802 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:57.767286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d9979-09df-4398-acde-3298412ad802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a42d9979-09df-4398-acde-3298412ad802 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:57.773687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:57.774646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:57.775695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 131/253] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:22:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:22:57.784689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c84aa-2036-4267-b7e0-9c7878853e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:57.785675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c84aa-2036-4267-b7e0-9c7878853e9f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:57.786159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c84aa-2036-4267-b7e0-9c7878853e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:57.786647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c84aa-2036-4267-b7e0-9c7878853e9f None admin] PATH_INFO: `/v3/users/981f5558161b450b8327c26a0cdabc8e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:57.790236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c84aa-2036-4267-b7e0-9c7878853e9f None admin] RBAC: Authorizing `identity:get_user(user_id=981f5558161b450b8327c26a0cdabc8e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:57.790485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d9979-09df-4398-acde-3298412ad802 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:57.792157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c84aa-2036-4267-b7e0-9c7878853e9f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:57.794352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 123/254] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:57 2022] GET /identity/v3/users/981f5558161b450b8327c26a0cdabc8e => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:57.819151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c84aa-2036-4267-b7e0-9c7878853e9f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:57.840554 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:57.921839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d9979-09df-4398-acde-3298412ad802 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:57.958141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbce52b-7888-4a31-920c-d8b986797d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:57.958759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbce52b-7888-4a31-920c-d8b986797d92 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:57.959031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbce52b-7888-4a31-920c-d8b986797d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:57.959305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbce52b-7888-4a31-920c-d8b986797d92 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:57.960800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbce52b-7888-4a31-920c-d8b986797d92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:57.961165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbce52b-7888-4a31-920c-d8b986797d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:57.962589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbce52b-7888-4a31-920c-d8b986797d92 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:57.971859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 132/255] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:22:57 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:57.981817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbce52b-7888-4a31-920c-d8b986797d92 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:58.019459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c84aa-2036-4267-b7e0-9c7878853e9f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:58.075762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dc2eb-7140-4c43-8a13-db8815fa801e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:58.076769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dc2eb-7140-4c43-8a13-db8815fa801e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:58.077274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dc2eb-7140-4c43-8a13-db8815fa801e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:58.077765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dc2eb-7140-4c43-8a13-db8815fa801e None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:58.082446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dc2eb-7140-4c43-8a13-db8815fa801e None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:58.085273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dc2eb-7140-4c43-8a13-db8815fa801e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:58.087385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 124/256] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:57 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:58.088820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab2a34-9305-4234-9aea-5fb97c66d30b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:58.089364 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab2a34-9305-4234-9aea-5fb97c66d30b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:58.089869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab2a34-9305-4234-9aea-5fb97c66d30b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:58.096737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 125/257] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:22:58 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:22:58.098016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab2a34-9305-4234-9aea-5fb97c66d30b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:58.112074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:58.116368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbce52b-7888-4a31-920c-d8b986797d92 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:58.156514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651076f-7d79-4e65-bb06-170b6b258803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:58.157237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651076f-7d79-4e65-bb06-170b6b258803 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:58.157541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651076f-7d79-4e65-bb06-170b6b258803 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:58.157931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651076f-7d79-4e65-bb06-170b6b258803 None admin] PATH_INFO: `/v3/users/5c16680130a9444bb448a7f20de5ac52` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:58.159134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651076f-7d79-4e65-bb06-170b6b258803 None admin] RBAC: Authorizing `identity:get_user(user_id=5c16680130a9444bb448a7f20de5ac52)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:58.160748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651076f-7d79-4e65-bb06-170b6b258803 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:58.162618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 133/258] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:57 2022] GET /identity/v3/users/5c16680130a9444bb448a7f20de5ac52 => generated 284 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:58.163626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c11f9b-80db-47ee-80ea-5ebf92479430 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:58.163949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c11f9b-80db-47ee-80ea-5ebf92479430 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:58.164174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c11f9b-80db-47ee-80ea-5ebf92479430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:58.183883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e0c11f9b-80db-47ee-80ea-5ebf92479430 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:22:58.185119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e0c11f9b-80db-47ee-80ea-5ebf92479430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:22:58.237600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:58.289923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 134/259] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:22:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:22:58.290814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c11f9b-80db-47ee-80ea-5ebf92479430 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:58.292900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab2a34-9305-4234-9aea-5fb97c66d30b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:58.335570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df556c-8dd1-4b90-9755-72aa0d11ce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:58.336598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df556c-8dd1-4b90-9755-72aa0d11ce05 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:58.336888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df556c-8dd1-4b90-9755-72aa0d11ce05 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:58.337164 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df556c-8dd1-4b90-9755-72aa0d11ce05 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078/users/981f5558161b450b8327c26a0cdabc8e/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:58.340456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df556c-8dd1-4b90-9755-72aa0d11ce05 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d0f6555111843d2af0acf5759071078, user_id=981f5558161b450b8327c26a0cdabc8e, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:58.341960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df556c-8dd1-4b90-9755-72aa0d11ce05 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:58.374757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 126/260] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:22:58 2022] PUT /identity/v3/projects/4d0f6555111843d2af0acf5759071078/users/981f5558161b450b8327c26a0cdabc8e/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:22:58.472410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:58.478260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c11f9b-80db-47ee-80ea-5ebf92479430 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:58.526337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45492a76-7be2-4faa-9c53-9738efe17ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:58.526968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45492a76-7be2-4faa-9c53-9738efe17ffe None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:58.527251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45492a76-7be2-4faa-9c53-9738efe17ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:58.527529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45492a76-7be2-4faa-9c53-9738efe17ffe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:58.625639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45492a76-7be2-4faa-9c53-9738efe17ffe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:58.627201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45492a76-7be2-4faa-9c53-9738efe17ffe None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:58.772679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:58.779377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 135/261] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:22:58.783225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df556c-8dd1-4b90-9755-72aa0d11ce05 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:58.819238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45492a76-7be2-4faa-9c53-9738efe17ffe None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:58.956326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df556c-8dd1-4b90-9755-72aa0d11ce05 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:58.986840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:59.015809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45492a76-7be2-4faa-9c53-9738efe17ffe None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:59.032478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633d3dd-b9a7-4dcb-aec6-735769e18759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:59.033122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633d3dd-b9a7-4dcb-aec6-735769e18759 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:59.033395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633d3dd-b9a7-4dcb-aec6-735769e18759 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:59.033701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633d3dd-b9a7-4dcb-aec6-735769e18759 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:59.036607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633d3dd-b9a7-4dcb-aec6-735769e18759 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:59.038901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633d3dd-b9a7-4dcb-aec6-735769e18759 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:59.042975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 127/262] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:22:58 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:59.048568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633d3dd-b9a7-4dcb-aec6-735769e18759 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:59.094928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbce9c3-2334-4ef9-a1d3-fd06855e9660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:59.095575 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbce9c3-2334-4ef9-a1d3-fd06855e9660 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:59.095851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbce9c3-2334-4ef9-a1d3-fd06855e9660 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:59.096117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbce9c3-2334-4ef9-a1d3-fd06855e9660 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:59.114402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbce9c3-2334-4ef9-a1d3-fd06855e9660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-dcbce9c3-2334-4ef9-a1d3-fd06855e9660 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:22:59.121077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:22:59.122176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:22:59.122960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:22:59.125833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 136/263] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:22:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:22:59.140398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbce9c3-2334-4ef9-a1d3-fd06855e9660 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:59.258024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:59.268260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633d3dd-b9a7-4dcb-aec6-735769e18759 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:59.318846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a026e47-c840-495d-bd28-970ffe370078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:59.319962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a026e47-c840-495d-bd28-970ffe370078 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:59.320617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a026e47-c840-495d-bd28-970ffe370078 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:59.321099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a026e47-c840-495d-bd28-970ffe370078 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:59.322078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a026e47-c840-495d-bd28-970ffe370078 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=4d0f6555111843d2af0acf5759071078, user.id=5c16680130a9444bb448a7f20de5ac52) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:59.322598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a026e47-c840-495d-bd28-970ffe370078 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:59.324410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a026e47-c840-495d-bd28-970ffe370078 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:59.332371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 128/264] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:22:59 2022] GET /identity/v3/role_assignments?user.id=5c16680130a9444bb448a7f20de5ac52&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 610 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:22:59.375766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbce9c3-2334-4ef9-a1d3-fd06855e9660 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:59.430972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28084a-7a3c-487d-b07d-afa3c79f7fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:59.431594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28084a-7a3c-487d-b07d-afa3c79f7fef None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:59.431874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28084a-7a3c-487d-b07d-afa3c79f7fef None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:59.432145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28084a-7a3c-487d-b07d-afa3c79f7fef None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:59.432921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28084a-7a3c-487d-b07d-afa3c79f7fef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:22:59.433226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28084a-7a3c-487d-b07d-afa3c79f7fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:59.434844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28084a-7a3c-487d-b07d-afa3c79f7fef None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:59.449769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 137/265] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:22:59 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:59.455534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a026e47-c840-495d-bd28-970ffe370078 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:59.523127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:59.588076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a026e47-c840-495d-bd28-970ffe370078 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:59.626226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d7011-8f33-4f25-8a3d-41806520da0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:59.627233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d7011-8f33-4f25-8a3d-41806520da0f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:59.627745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d7011-8f33-4f25-8a3d-41806520da0f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:59.628219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d7011-8f33-4f25-8a3d-41806520da0f None admin] PATH_INFO: `/v3/users/5c16680130a9444bb448a7f20de5ac52` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:59.629732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d7011-8f33-4f25-8a3d-41806520da0f None admin] RBAC: Authorizing `identity:get_user(user_id=5c16680130a9444bb448a7f20de5ac52)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:59.632031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d7011-8f33-4f25-8a3d-41806520da0f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:22:59.634357 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 129/266] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:22:59 2022] GET /identity/v3/users/5c16680130a9444bb448a7f20de5ac52 => generated 284 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:22:59.640471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28084a-7a3c-487d-b07d-afa3c79f7fef None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:22:59.802298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28084a-7a3c-487d-b07d-afa3c79f7fef None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:22:59.837224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:22:59.855552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834bfe2-7575-4cd7-8da9-b3e810fe71b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:22:59.856185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834bfe2-7575-4cd7-8da9-b3e810fe71b4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:22:59.856467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834bfe2-7575-4cd7-8da9-b3e810fe71b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:22:59.856737 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834bfe2-7575-4cd7-8da9-b3e810fe71b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:22:59.955957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834bfe2-7575-4cd7-8da9-b3e810fe71b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:22:59.957475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834bfe2-7575-4cd7-8da9-b3e810fe71b4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:00.097982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828e7e6-88ad-438d-b9d7-d41e3847a816 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:00.098171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828e7e6-88ad-438d-b9d7-d41e3847a816 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:00.098440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828e7e6-88ad-438d-b9d7-d41e3847a816 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:00.100093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 130/267] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:00.105516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590863-d96c-43bb-bdd2-b860b88d979a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:00.105833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590863-d96c-43bb-bdd2-b860b88d979a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:00.106110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590863-d96c-43bb-bdd2-b860b88d979a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:00.138622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-39590863-d96c-43bb-bdd2-b860b88d979a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:00.139845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-39590863-d96c-43bb-bdd2-b860b88d979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:00.159455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 138/268] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:22:59 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:00.167039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834bfe2-7575-4cd7-8da9-b3e810fe71b4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:00.198489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:00.246104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 131/269] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:23:00.250156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd71da-748e-4a6e-a838-fe92fa6c0cc1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:00.250587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd71da-748e-4a6e-a838-fe92fa6c0cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:00.250981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd71da-748e-4a6e-a838-fe92fa6c0cc1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:00.252741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 132/270] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:00 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:00.259426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08245e4-59f6-4988-ae1e-b1f6cd7a6345 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:00.260100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08245e4-59f6-4988-ae1e-b1f6cd7a6345 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:00.261919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08245e4-59f6-4988-ae1e-b1f6cd7a6345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:00.298504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c08245e4-59f6-4988-ae1e-b1f6cd7a6345 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:00.300839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c08245e4-59f6-4988-ae1e-b1f6cd7a6345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:00.324564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834bfe2-7575-4cd7-8da9-b3e810fe71b4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:00.372337 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:00.380282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77883a8-1b78-42c1-a175-ddb8dd5e0a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:00.384273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77883a8-1b78-42c1-a175-ddb8dd5e0a8a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:00.385804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77883a8-1b78-42c1-a175-ddb8dd5e0a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:00.386517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77883a8-1b78-42c1-a175-ddb8dd5e0a8a None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:00.390464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77883a8-1b78-42c1-a175-ddb8dd5e0a8a None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:00.394361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77883a8-1b78-42c1-a175-ddb8dd5e0a8a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:00.401371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 139/271] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:00 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:00.402956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40769b-2443-4374-94f5-31cd1f969f15 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:00.403247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40769b-2443-4374-94f5-31cd1f969f15 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:00.403499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40769b-2443-4374-94f5-31cd1f969f15 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:00.405022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 140/272] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:00.414774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb40769b-2443-4374-94f5-31cd1f969f15 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:00.424478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 133/273] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:00.425524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b58d3-c4df-47b9-93d5-244c3056687a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:00.425830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b58d3-c4df-47b9-93d5-244c3056687a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:00.426102 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b58d3-c4df-47b9-93d5-244c3056687a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:00.456961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-165b58d3-c4df-47b9-93d5-244c3056687a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:00.458180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-165b58d3-c4df-47b9-93d5-244c3056687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:00.623648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:00.636827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 134/274] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:00.672011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb40769b-2443-4374-94f5-31cd1f969f15 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:00.733820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ce26a-fe76-4c30-a63b-1f81da6ef4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:00.734741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ce26a-fe76-4c30-a63b-1f81da6ef4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:00.735209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ce26a-fe76-4c30-a63b-1f81da6ef4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:00.735674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ce26a-fe76-4c30-a63b-1f81da6ef4a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:00.736941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ce26a-fe76-4c30-a63b-1f81da6ef4a1 None admin] RBAC: Adding query filter params (role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=4d0f6555111843d2af0acf5759071078, user.id=5c16680130a9444bb448a7f20de5ac52) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:00.737774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ce26a-fe76-4c30-a63b-1f81da6ef4a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:00.740367 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ce26a-fe76-4c30-a63b-1f81da6ef4a1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:00.752179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 141/275] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:00 2022] GET /identity/v3/role_assignments?user.id=5c16680130a9444bb448a7f20de5ac52&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=5dc7056ddc154b81939a17474a64f546 => generated 610 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:00.760073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b58d3-c4df-47b9-93d5-244c3056687a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:00.810211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ce26a-fe76-4c30-a63b-1f81da6ef4a1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:00.971957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b58d3-c4df-47b9-93d5-244c3056687a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:00.979020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:01.039990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ce26a-fe76-4c30-a63b-1f81da6ef4a1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:01.046260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6d0f-96d8-48a9-b8bf-db52fd329dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:01.047443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6d0f-96d8-48a9-b8bf-db52fd329dad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.047735 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6d0f-96d8-48a9-b8bf-db52fd329dad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.049428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6d0f-96d8-48a9-b8bf-db52fd329dad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.051042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a6d0f-96d8-48a9-b8bf-db52fd329dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:01.052909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a6d0f-96d8-48a9-b8bf-db52fd329dad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:01.055405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 135/276] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:00 2022] GET /identity/v3/domains/default => generated 200 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:01.058313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6d0f-96d8-48a9-b8bf-db52fd329dad None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:01.103534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feeaaba-07ef-4f36-bdfa-a8b86603dabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:01.104226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feeaaba-07ef-4f36-bdfa-a8b86603dabc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.104506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feeaaba-07ef-4f36-bdfa-a8b86603dabc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.104781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feeaaba-07ef-4f36-bdfa-a8b86603dabc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.118783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feeaaba-07ef-4f36-bdfa-a8b86603dabc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4feeaaba-07ef-4f36-bdfa-a8b86603dabc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:01.126400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:01.127374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:01.128217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 142/277] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:01.128878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feeaaba-07ef-4f36-bdfa-a8b86603dabc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:01.150793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:01.195500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6d0f-96d8-48a9-b8bf-db52fd329dad None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:01.233185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb983c6e-d176-400f-8d52-69dc8695aaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:01.233859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb983c6e-d176-400f-8d52-69dc8695aaac None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.234135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb983c6e-d176-400f-8d52-69dc8695aaac None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.234400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb983c6e-d176-400f-8d52-69dc8695aaac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.246715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb983c6e-d176-400f-8d52-69dc8695aaac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-fb983c6e-d176-400f-8d52-69dc8695aaac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:01.253455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:01.254547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:01.255339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:01.255964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feeaaba-07ef-4f36-bdfa-a8b86603dabc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:01.256035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 136/278] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:01.256035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb983c6e-d176-400f-8d52-69dc8695aaac None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:01.294002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:01.299641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed79e4-ae71-461a-aaf8-b2042a107337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:01.300328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed79e4-ae71-461a-aaf8-b2042a107337 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.300609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed79e4-ae71-461a-aaf8-b2042a107337 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.300882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed79e4-ae71-461a-aaf8-b2042a107337 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.301531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed79e4-ae71-461a-aaf8-b2042a107337 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:01.303024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed79e4-ae71-461a-aaf8-b2042a107337 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:01.333158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 143/279] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:23:01 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:23:01.335282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed79e4-ae71-461a-aaf8-b2042a107337 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:01.407817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb983c6e-d176-400f-8d52-69dc8695aaac None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:01.455862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:01.466363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f078a9b-4134-437a-bbab-b242c684e558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:01.468092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f078a9b-4134-437a-bbab-b242c684e558 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.468971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f078a9b-4134-437a-bbab-b242c684e558 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.469549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f078a9b-4134-437a-bbab-b242c684e558 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.470579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f078a9b-4134-437a-bbab-b242c684e558 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:01.471123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f078a9b-4134-437a-bbab-b242c684e558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:01.472768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f078a9b-4134-437a-bbab-b242c684e558 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:01.486129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 137/280] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:01 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:01.494980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f078a9b-4134-437a-bbab-b242c684e558 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:01.512951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed79e4-ae71-461a-aaf8-b2042a107337 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:01.555606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101274cc-a885-422d-b49a-48bfed86a46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:01.556210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101274cc-a885-422d-b49a-48bfed86a46f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.556494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101274cc-a885-422d-b49a-48bfed86a46f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.556761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101274cc-a885-422d-b49a-48bfed86a46f None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.557444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101274cc-a885-422d-b49a-48bfed86a46f None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:01.557780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101274cc-a885-422d-b49a-48bfed86a46f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:01.559225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101274cc-a885-422d-b49a-48bfed86a46f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:01.568864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 144/281] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:01 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:01.570854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7457c0-d9f0-4986-bf6d-42000b905b7a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.571188 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7457c0-d9f0-4986-bf6d-42000b905b7a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.571471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7457c0-d9f0-4986-bf6d-42000b905b7a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.572757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 145/282] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:01.578111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7457c0-d9f0-4986-bf6d-42000b905b7a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:01.626166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:01.664231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f078a9b-4134-437a-bbab-b242c684e558 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:01.728546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039ec71-389b-42ff-9179-468ddf669bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:01.729745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039ec71-389b-42ff-9179-468ddf669bdf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.730892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039ec71-389b-42ff-9179-468ddf669bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.730892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039ec71-389b-42ff-9179-468ddf669bdf None admin] PATH_INFO: `/v3/users/01a02236d2cf4ec0beab6f4214b37940` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.732511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039ec71-389b-42ff-9179-468ddf669bdf None admin] RBAC: Authorizing `identity:get_user(user_id=01a02236d2cf4ec0beab6f4214b37940)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:01.734297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7457c0-d9f0-4986-bf6d-42000b905b7a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:01.735260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039ec71-389b-42ff-9179-468ddf669bdf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:01.739035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 138/283] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:23:01 2022] GET /identity/v3/users/01a02236d2cf4ec0beab6f4214b37940 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:01.740658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66342e-0b31-49fd-94f9-1ad24b0faf67 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.741115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66342e-0b31-49fd-94f9-1ad24b0faf67 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.741523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66342e-0b31-49fd-94f9-1ad24b0faf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.768890 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:01.774975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8f66342e-0b31-49fd-94f9-1ad24b0faf67 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:01.777206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8f66342e-0b31-49fd-94f9-1ad24b0faf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:01.778025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca1ecc-21a7-4ebf-988e-75abc8c39d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:01.778931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca1ecc-21a7-4ebf-988e-75abc8c39d00 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.779425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca1ecc-21a7-4ebf-988e-75abc8c39d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.779961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca1ecc-21a7-4ebf-988e-75abc8c39d00 None admin] PATH_INFO: `/v3/users/b162f00c4c5b40e7858f82dce0708608` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.797282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca1ecc-21a7-4ebf-988e-75abc8c39d00 None admin] RBAC: Authorizing `identity:get_user(user_id=b162f00c4c5b40e7858f82dce0708608)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:01.798822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca1ecc-21a7-4ebf-988e-75abc8c39d00 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:01.800942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 146/284] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:23:01 2022] GET /identity/v3/users/b162f00c4c5b40e7858f82dce0708608 => generated 294 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:01.801787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca1ecc-21a7-4ebf-988e-75abc8c39d00 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:01.895476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 139/285] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:01.897599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66342e-0b31-49fd-94f9-1ad24b0faf67 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:01.913395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca1ecc-21a7-4ebf-988e-75abc8c39d00 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:01.950677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f140d3-37c8-406b-a03e-2f0b0de4e337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:01.951321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f140d3-37c8-406b-a03e-2f0b0de4e337 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:01.951605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f140d3-37c8-406b-a03e-2f0b0de4e337 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:01.951885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f140d3-37c8-406b-a03e-2f0b0de4e337 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:01.953124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f140d3-37c8-406b-a03e-2f0b0de4e337 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:01.954688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f140d3-37c8-406b-a03e-2f0b0de4e337 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:01.956331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 147/286] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:01 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:01.962374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f140d3-37c8-406b-a03e-2f0b0de4e337 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:02.042036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:02.058087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:02.074724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66342e-0b31-49fd-94f9-1ad24b0faf67 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:02.165365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c773ce-477a-467c-9788-adc70bb5ef69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:02.166401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c773ce-477a-467c-9788-adc70bb5ef69 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:02.166879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c773ce-477a-467c-9788-adc70bb5ef69 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:02.167336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c773ce-477a-467c-9788-adc70bb5ef69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:02.168646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c773ce-477a-467c-9788-adc70bb5ef69 None admin] RBAC: Adding query filter params (role.id=90949fa670c2407ca92b16156e52f47f, user.id=b162f00c4c5b40e7858f82dce0708608, scope.system=all) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:02.169187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c773ce-477a-467c-9788-adc70bb5ef69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:02.174725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f140d3-37c8-406b-a03e-2f0b0de4e337 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:02.176403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c773ce-477a-467c-9788-adc70bb5ef69 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:02.182894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 140/287] 10.222.0.100 () {56 vars in 1313 bytes} [Thu Sep 1 17:23:01 2022] GET /identity/v3/role_assignments?user.id=b162f00c4c5b40e7858f82dce0708608&scope.system=all&role.id=90949fa670c2407ca92b16156e52f47f => generated 227 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:02.208199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c773ce-477a-467c-9788-adc70bb5ef69 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:02.237277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39189df-f58d-475c-a223-75a78e9bb9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:02.238330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39189df-f58d-475c-a223-75a78e9bb9ab None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:02.239047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39189df-f58d-475c-a223-75a78e9bb9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:02.240524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39189df-f58d-475c-a223-75a78e9bb9ab None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078/users/01a02236d2cf4ec0beab6f4214b37940/roles/e272ecbda33d4b3e9308242d2d4ab302` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:02.244602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39189df-f58d-475c-a223-75a78e9bb9ab None admin] RBAC: Authorizing `identity:create_grant(project_id=4d0f6555111843d2af0acf5759071078, user_id=01a02236d2cf4ec0beab6f4214b37940, role_id=e272ecbda33d4b3e9308242d2d4ab302)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:02.246711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39189df-f58d-475c-a223-75a78e9bb9ab None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:02.274146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 148/288] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:23:01 2022] PUT /identity/v3/projects/4d0f6555111843d2af0acf5759071078/users/01a02236d2cf4ec0beab6f4214b37940/roles/e272ecbda33d4b3e9308242d2d4ab302 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:02.360194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c773ce-477a-467c-9788-adc70bb5ef69 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:02.408334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f1680-6137-48d9-900f-d9e8aefb5097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:02.409299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f1680-6137-48d9-900f-d9e8aefb5097 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:02.409826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f1680-6137-48d9-900f-d9e8aefb5097 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:02.410274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f1680-6137-48d9-900f-d9e8aefb5097 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:02.426095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f1680-6137-48d9-900f-d9e8aefb5097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5a1f1680-6137-48d9-900f-d9e8aefb5097 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:02.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:02.432448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:02.433272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:02.436045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 141/289] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:02.443310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39189df-f58d-475c-a223-75a78e9bb9ab None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:02.564976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39189df-f58d-475c-a223-75a78e9bb9ab None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:02.589727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:02.609401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8af8d-3be5-409e-aa8d-28efe6de8ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:02.610041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8af8d-3be5-409e-aa8d-28efe6de8ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:02.610373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8af8d-3be5-409e-aa8d-28efe6de8ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:02.610591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8af8d-3be5-409e-aa8d-28efe6de8ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:02.611257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8af8d-3be5-409e-aa8d-28efe6de8ba2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:02.611553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8af8d-3be5-409e-aa8d-28efe6de8ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:02.612929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8af8d-3be5-409e-aa8d-28efe6de8ba2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:02.622109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 149/290] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:02 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:02.627708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f1680-6137-48d9-900f-d9e8aefb5097 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:02.655397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:02.761738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f1680-6137-48d9-900f-d9e8aefb5097 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:02.804845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24574e70-4f6f-4aed-8e00-ace97f9c95f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:02.805498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24574e70-4f6f-4aed-8e00-ace97f9c95f5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:02.805802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24574e70-4f6f-4aed-8e00-ace97f9c95f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:02.806076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24574e70-4f6f-4aed-8e00-ace97f9c95f5 None admin] PATH_INFO: `/v3/users/981f5558161b450b8327c26a0cdabc8e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:02.807258 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24574e70-4f6f-4aed-8e00-ace97f9c95f5 None admin] RBAC: Authorizing `identity:get_user(user_id=981f5558161b450b8327c26a0cdabc8e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:02.808728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24574e70-4f6f-4aed-8e00-ace97f9c95f5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:02.810515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 142/291] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:23:02 2022] GET /identity/v3/users/981f5558161b450b8327c26a0cdabc8e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:02.816769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8af8d-3be5-409e-aa8d-28efe6de8ba2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:02.985799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8af8d-3be5-409e-aa8d-28efe6de8ba2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:03.025745 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e78cc-cc3c-4621-8450-f9204d45f42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:03.026373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e78cc-cc3c-4621-8450-f9204d45f42d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:03.026653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e78cc-cc3c-4621-8450-f9204d45f42d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:03.026927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e78cc-cc3c-4621-8450-f9204d45f42d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:03.093527 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:03.119045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e78cc-cc3c-4621-8450-f9204d45f42d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:03.120580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e78cc-cc3c-4621-8450-f9204d45f42d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:03.241891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 150/292] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:03.247630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24574e70-4f6f-4aed-8e00-ace97f9c95f5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:03.370875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:03.406361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24574e70-4f6f-4aed-8e00-ace97f9c95f5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:03.467780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c3507-f9bf-4f0e-8897-c7ac2fcbabd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:03.468438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c3507-f9bf-4f0e-8897-c7ac2fcbabd0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:03.468717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c3507-f9bf-4f0e-8897-c7ac2fcbabd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:03.468999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c3507-f9bf-4f0e-8897-c7ac2fcbabd0 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:03.474377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c3507-f9bf-4f0e-8897-c7ac2fcbabd0 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:03.476506 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c3507-f9bf-4f0e-8897-c7ac2fcbabd0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:03.478710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 143/293] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:03 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:03.484202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e78cc-cc3c-4621-8450-f9204d45f42d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:03.577153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:03.606058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e78cc-cc3c-4621-8450-f9204d45f42d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:03.643102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b139e-f61e-4bbb-a7d7-8a542eeadf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:03.643706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b139e-f61e-4bbb-a7d7-8a542eeadf36 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:03.643983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b139e-f61e-4bbb-a7d7-8a542eeadf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:03.644251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b139e-f61e-4bbb-a7d7-8a542eeadf36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:03.644973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b139e-f61e-4bbb-a7d7-8a542eeadf36 None admin] RBAC: Adding query filter params (role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=4d0f6555111843d2af0acf5759071078, user.id=981f5558161b450b8327c26a0cdabc8e) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:03.645272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b139e-f61e-4bbb-a7d7-8a542eeadf36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:03.646725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b139e-f61e-4bbb-a7d7-8a542eeadf36 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:03.655192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 151/294] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:03 2022] GET /identity/v3/role_assignments?user.id=981f5558161b450b8327c26a0cdabc8e&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=90949fa670c2407ca92b16156e52f47f => generated 610 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:04.076457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aeae50-b958-453a-bd5b-17cf6cb85da6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.076966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aeae50-b958-453a-bd5b-17cf6cb85da6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.077072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aeae50-b958-453a-bd5b-17cf6cb85da6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.078632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 144/295] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:04.085195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09254c-24f0-4add-96bc-5ef867cc0238 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.085890 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09254c-24f0-4add-96bc-5ef867cc0238 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.085890 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09254c-24f0-4add-96bc-5ef867cc0238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.108016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3d09254c-24f0-4add-96bc-5ef867cc0238 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:04.109219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3d09254c-24f0-4add-96bc-5ef867cc0238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:04.186526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 152/296] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:04.187066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43eeb-72c8-41aa-baaf-4362cb78e681 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.187355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43eeb-72c8-41aa-baaf-4362cb78e681 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.187628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43eeb-72c8-41aa-baaf-4362cb78e681 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.189234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 145/297] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:04.194613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4bede-ecff-42b8-9c53-be105f04e1f1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.194904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4bede-ecff-42b8-9c53-be105f04e1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.195169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4bede-ecff-42b8-9c53-be105f04e1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.214984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-acc4bede-ecff-42b8-9c53-be105f04e1f1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:04.216164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-acc4bede-ecff-42b8-9c53-be105f04e1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:04.291569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 153/298] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:04.460428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c43eeb-72c8-41aa-baaf-4362cb78e681 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:04.563363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4bede-ecff-42b8-9c53-be105f04e1f1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:04.565612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:04.567605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c43eeb-72c8-41aa-baaf-4362cb78e681 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:04.605982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c2034-ae9c-4b03-aa45-b34258dd820d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:04.606624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c2034-ae9c-4b03-aa45-b34258dd820d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.609621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c2034-ae9c-4b03-aa45-b34258dd820d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.609621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c2034-ae9c-4b03-aa45-b34258dd820d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.624808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c2034-ae9c-4b03-aa45-b34258dd820d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-459c2034-ae9c-4b03-aa45-b34258dd820d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:04.630027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:04.630948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:04.632439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 146/299] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:04.637886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c2034-ae9c-4b03-aa45-b34258dd820d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:04.652763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:04.686891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4bede-ecff-42b8-9c53-be105f04e1f1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:04.724357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b10ca-cd87-4aa1-bdc3-d57b3c4fd73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:04.726120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b10ca-cd87-4aa1-bdc3-d57b3c4fd73f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.726840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b10ca-cd87-4aa1-bdc3-d57b3c4fd73f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.727318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b10ca-cd87-4aa1-bdc3-d57b3c4fd73f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.728743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b10ca-cd87-4aa1-bdc3-d57b3c4fd73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:04.730473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b10ca-cd87-4aa1-bdc3-d57b3c4fd73f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:04.732649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 154/300] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:04 2022] GET /identity/v3/domains/default => generated 200 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:04.738872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b10ca-cd87-4aa1-bdc3-d57b3c4fd73f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:04.763245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c2034-ae9c-4b03-aa45-b34258dd820d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:04.803832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0860f-fb26-4a35-8062-ecd85f9221c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:04.804758 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0860f-fb26-4a35-8062-ecd85f9221c1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.805035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0860f-fb26-4a35-8062-ecd85f9221c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.805307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0860f-fb26-4a35-8062-ecd85f9221c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.806014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0860f-fb26-4a35-8062-ecd85f9221c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:04.806310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0860f-fb26-4a35-8062-ecd85f9221c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:04.807687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0860f-fb26-4a35-8062-ecd85f9221c1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:04.816227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 147/301] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:04 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:04.817559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1d51d-3d01-4b14-9e68-131a8abd4261 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.817872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1d51d-3d01-4b14-9e68-131a8abd4261 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.818139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1d51d-3d01-4b14-9e68-131a8abd4261 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.821958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 148/302] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:04 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:04.823216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0531c-751a-4290-acef-456df04e6d40 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.823501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0531c-751a-4290-acef-456df04e6d40 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.823818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0531c-751a-4290-acef-456df04e6d40 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.825254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 149/303] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:04.825983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0531c-751a-4290-acef-456df04e6d40 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:04.827187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:04.860642 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b10ca-cd87-4aa1-bdc3-d57b3c4fd73f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:04.895837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355e326-9b5f-49cc-ac3c-bd67aa7437c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:04.896442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355e326-9b5f-49cc-ac3c-bd67aa7437c0 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.896724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355e326-9b5f-49cc-ac3c-bd67aa7437c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.897731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355e326-9b5f-49cc-ac3c-bd67aa7437c0 None admin] PATH_INFO: `/v3/groups` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.898622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355e326-9b5f-49cc-ac3c-bd67aa7437c0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:04.900066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355e326-9b5f-49cc-ac3c-bd67aa7437c0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:04.916223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:04.925394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 155/304] 10.222.0.100 () {60 vars in 1139 bytes} [Thu Sep 1 17:23:04 2022] POST /identity/v3/groups => generated 230 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:23:04.926953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39235b-86dd-4d7f-b99d-3138fe6374d6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:04.927234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39235b-86dd-4d7f-b99d-3138fe6374d6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:04.927633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39235b-86dd-4d7f-b99d-3138fe6374d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:04.945058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0531c-751a-4290-acef-456df04e6d40 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:04.948758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cd39235b-86dd-4d7f-b99d-3138fe6374d6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:04.950049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cd39235b-86dd-4d7f-b99d-3138fe6374d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:05.005599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac45f05-fbeb-40d5-854e-93d20ad6dea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:05.006913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45f05-fbeb-40d5-854e-93d20ad6dea2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.007382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45f05-fbeb-40d5-854e-93d20ad6dea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.007831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45f05-fbeb-40d5-854e-93d20ad6dea2 None admin] PATH_INFO: `/v3/users/42de788ae47345198b9b6a679a2d2423` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.033786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac45f05-fbeb-40d5-854e-93d20ad6dea2 None admin] RBAC: Authorizing `identity:get_user(user_id=42de788ae47345198b9b6a679a2d2423)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:05.035505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac45f05-fbeb-40d5-854e-93d20ad6dea2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:05.043602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 150/305] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:23:04 2022] GET /identity/v3/users/42de788ae47345198b9b6a679a2d2423 => generated 294 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:05.043602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b562ab8-24de-4e66-a9ad-e9e1f3701a56 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.043602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b562ab8-24de-4e66-a9ad-e9e1f3701a56 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.043602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b562ab8-24de-4e66-a9ad-e9e1f3701a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.059755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3b562ab8-24de-4e66-a9ad-e9e1f3701a56 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:05.060990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3b562ab8-24de-4e66-a9ad-e9e1f3701a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:05.073622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 156/306] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:05.073622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39235b-86dd-4d7f-b99d-3138fe6374d6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:05.159776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:05.163599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 151/307] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:05.215531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39235b-86dd-4d7f-b99d-3138fe6374d6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:05.254093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d38628-09bf-494f-8ab9-a7b370dcaf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:05.254701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d38628-09bf-494f-8ab9-a7b370dcaf14 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.254982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d38628-09bf-494f-8ab9-a7b370dcaf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.255251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d38628-09bf-494f-8ab9-a7b370dcaf14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.256042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d38628-09bf-494f-8ab9-a7b370dcaf14 None admin] RBAC: Adding query filter params (role.id=e272ecbda33d4b3e9308242d2d4ab302, user.id=42de788ae47345198b9b6a679a2d2423, scope.system=all) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:05.256323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d38628-09bf-494f-8ab9-a7b370dcaf14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:05.257801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d38628-09bf-494f-8ab9-a7b370dcaf14 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:05.264102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 157/308] 10.222.0.100 () {56 vars in 1313 bytes} [Thu Sep 1 17:23:05 2022] GET /identity/v3/role_assignments?user.id=42de788ae47345198b9b6a679a2d2423&scope.system=all&role.id=e272ecbda33d4b3e9308242d2d4ab302 => generated 227 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:05.359377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b562ab8-24de-4e66-a9ad-e9e1f3701a56 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:05.429604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:05.456706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d38628-09bf-494f-8ab9-a7b370dcaf14 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:05.484663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b562ab8-24de-4e66-a9ad-e9e1f3701a56 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:05.531436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13609bf-dbcc-4c47-913b-aff4540e925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:05.531997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13609bf-dbcc-4c47-913b-aff4540e925d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.532417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13609bf-dbcc-4c47-913b-aff4540e925d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.532690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13609bf-dbcc-4c47-913b-aff4540e925d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.546634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13609bf-dbcc-4c47-913b-aff4540e925d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b13609bf-dbcc-4c47-913b-aff4540e925d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:05.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:05.554676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:05.555655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:05.562949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 152/309] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:05.564769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13609bf-dbcc-4c47-913b-aff4540e925d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:05.576038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d38628-09bf-494f-8ab9-a7b370dcaf14 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:05.606270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:05.618701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41604f92-ed8c-4a06-b548-bb07b2e74656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:05.619480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41604f92-ed8c-4a06-b548-bb07b2e74656 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.619858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41604f92-ed8c-4a06-b548-bb07b2e74656 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.620226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41604f92-ed8c-4a06-b548-bb07b2e74656 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.630849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41604f92-ed8c-4a06-b548-bb07b2e74656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-41604f92-ed8c-4a06-b548-bb07b2e74656 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:05.636535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:05.637660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:05.638652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:05.640532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 158/310] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:05.647729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41604f92-ed8c-4a06-b548-bb07b2e74656 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:05.700725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13609bf-dbcc-4c47-913b-aff4540e925d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:05.724673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:05.742844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ee693-5210-4b34-a8aa-060cbda5e04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:05.743696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ee693-5210-4b34-a8aa-060cbda5e04a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.744158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ee693-5210-4b34-a8aa-060cbda5e04a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.744569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ee693-5210-4b34-a8aa-060cbda5e04a None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.745408 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ee693-5210-4b34-a8aa-060cbda5e04a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:05.745902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ee693-5210-4b34-a8aa-060cbda5e04a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:05.747576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ee693-5210-4b34-a8aa-060cbda5e04a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:05.759177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 153/311] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:05 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:05.763844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41604f92-ed8c-4a06-b548-bb07b2e74656 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:05.764502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ee693-5210-4b34-a8aa-060cbda5e04a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:05.811280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17056bc2-1575-468b-b6f1-cb098f68e0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:05.812777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17056bc2-1575-468b-b6f1-cb098f68e0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.813724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17056bc2-1575-468b-b6f1-cb098f68e0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.814380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17056bc2-1575-468b-b6f1-cb098f68e0b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.815378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17056bc2-1575-468b-b6f1-cb098f68e0b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:05.815860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17056bc2-1575-468b-b6f1-cb098f68e0b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:05.817808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17056bc2-1575-468b-b6f1-cb098f68e0b5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:05.829870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 159/312] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:05 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:05.839023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17056bc2-1575-468b-b6f1-cb098f68e0b5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:05.914164 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ee693-5210-4b34-a8aa-060cbda5e04a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:05.915765 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:05.982334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec9dfa-acbc-4b85-b40d-b3f60f60af04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:05.983372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec9dfa-acbc-4b85-b40d-b3f60f60af04 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:05.983840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec9dfa-acbc-4b85-b40d-b3f60f60af04 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:05.984375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec9dfa-acbc-4b85-b40d-b3f60f60af04 None admin] PATH_INFO: `/v3/users/01a02236d2cf4ec0beab6f4214b37940` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:05.986306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec9dfa-acbc-4b85-b40d-b3f60f60af04 None admin] RBAC: Authorizing `identity:get_user(user_id=01a02236d2cf4ec0beab6f4214b37940)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:05.989192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec9dfa-acbc-4b85-b40d-b3f60f60af04 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:05.999481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 154/313] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:23:05 2022] GET /identity/v3/users/01a02236d2cf4ec0beab6f4214b37940 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:06.002610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec9dfa-acbc-4b85-b40d-b3f60f60af04 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:06.028899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17056bc2-1575-468b-b6f1-cb098f68e0b5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:06.088038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:06.099992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15fd04-0ef6-4240-89fe-3143c90c7854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:06.100999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15fd04-0ef6-4240-89fe-3143c90c7854 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:06.101939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15fd04-0ef6-4240-89fe-3143c90c7854 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:06.102412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15fd04-0ef6-4240-89fe-3143c90c7854 None admin] PATH_INFO: `/v3/users/b162f00c4c5b40e7858f82dce0708608` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:06.104600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15fd04-0ef6-4240-89fe-3143c90c7854 None admin] RBAC: Authorizing `identity:get_user(user_id=b162f00c4c5b40e7858f82dce0708608)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:06.107903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15fd04-0ef6-4240-89fe-3143c90c7854 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:06.113765 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 160/314] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:23:05 2022] GET /identity/v3/users/b162f00c4c5b40e7858f82dce0708608 => generated 294 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:06.119040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15fd04-0ef6-4240-89fe-3143c90c7854 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:06.170305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec9dfa-acbc-4b85-b40d-b3f60f60af04 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:06.223672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3712242-8909-4df7-b964-c94c885674b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:06.224283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3712242-8909-4df7-b964-c94c885674b1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:06.224565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3712242-8909-4df7-b964-c94c885674b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:06.225690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3712242-8909-4df7-b964-c94c885674b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:06.237201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15fd04-0ef6-4240-89fe-3143c90c7854 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:06.256124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:06.280017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf4b69-1a45-4b55-b9cc-3baae04d842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:06.281060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf4b69-1a45-4b55-b9cc-3baae04d842f None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:06.281537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf4b69-1a45-4b55-b9cc-3baae04d842f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:06.282130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf4b69-1a45-4b55-b9cc-3baae04d842f None admin] PATH_INFO: `/v3/system/users/b162f00c4c5b40e7858f82dce0708608/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:06.283780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf4b69-1a45-4b55-b9cc-3baae04d842f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b162f00c4c5b40e7858f82dce0708608, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:06.285835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf4b69-1a45-4b55-b9cc-3baae04d842f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:06.298781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 161/315] 10.222.0.100 () {58 vars in 1339 bytes} [Thu Sep 1 17:23:06 2022] PUT /identity/v3/system/users/b162f00c4c5b40e7858f82dce0708608/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:23:06.337380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3712242-8909-4df7-b964-c94c885674b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:06.339059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3712242-8909-4df7-b964-c94c885674b1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:06.397879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:06.464260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 155/316] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:06 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:23:06.466374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf4b69-1a45-4b55-b9cc-3baae04d842f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:06.574356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf4b69-1a45-4b55-b9cc-3baae04d842f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:06.610576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15137412-f029-4bc2-9873-da94e7b9f583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:06.611138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15137412-f029-4bc2-9873-da94e7b9f583 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:06.611420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15137412-f029-4bc2-9873-da94e7b9f583 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:06.611693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15137412-f029-4bc2-9873-da94e7b9f583 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:06.612822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15137412-f029-4bc2-9873-da94e7b9f583 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:06.614308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15137412-f029-4bc2-9873-da94e7b9f583 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:06.616862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 162/317] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:06 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:06.623669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3712242-8909-4df7-b964-c94c885674b1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:06.734910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3712242-8909-4df7-b964-c94c885674b1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:06.772409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8debe5-5ccc-46b5-aa51-0b8dd6eb70de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:06.773171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8debe5-5ccc-46b5-aa51-0b8dd6eb70de None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:06.773455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8debe5-5ccc-46b5-aa51-0b8dd6eb70de None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:06.773756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8debe5-5ccc-46b5-aa51-0b8dd6eb70de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:06.774499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8debe5-5ccc-46b5-aa51-0b8dd6eb70de None admin] RBAC: Adding query filter params (role.id=e272ecbda33d4b3e9308242d2d4ab302, scope.project.id=4d0f6555111843d2af0acf5759071078, user.id=01a02236d2cf4ec0beab6f4214b37940) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:06.774812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8debe5-5ccc-46b5-aa51-0b8dd6eb70de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:06.776575 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8debe5-5ccc-46b5-aa51-0b8dd6eb70de None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:06.789761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 156/318] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:06 2022] GET /identity/v3/role_assignments?user.id=01a02236d2cf4ec0beab6f4214b37940&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=e272ecbda33d4b3e9308242d2d4ab302 => generated 610 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:07.073757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247ff5-2847-4182-b5bb-aa2b47dfe06a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:07.074159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247ff5-2847-4182-b5bb-aa2b47dfe06a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:07.074333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247ff5-2847-4182-b5bb-aa2b47dfe06a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:07.075817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 163/319] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:07.081083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3269541-93c7-4d9c-b3cc-a9993e50bbb5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:07.081370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3269541-93c7-4d9c-b3cc-a9993e50bbb5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:07.081683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3269541-93c7-4d9c-b3cc-a9993e50bbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:07.101711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c3269541-93c7-4d9c-b3cc-a9993e50bbb5 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:07.102921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c3269541-93c7-4d9c-b3cc-a9993e50bbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:07.113440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:07.181203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 157/320] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:07.322165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60c302-d868-40ce-9fb7-058c7a9cd44a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:07.322683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60c302-d868-40ce-9fb7-058c7a9cd44a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:07.322968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60c302-d868-40ce-9fb7-058c7a9cd44a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:07.324548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 164/321] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:07.330027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f2f19-f41f-43bf-86a7-de6466915d05 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:07.330313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f2f19-f41f-43bf-86a7-de6466915d05 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:07.330585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f2f19-f41f-43bf-86a7-de6466915d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:07.349764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-093f2f19-f41f-43bf-86a7-de6466915d05 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:07.351785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-093f2f19-f41f-43bf-86a7-de6466915d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:07.433987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 158/322] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:07.449416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60c302-d868-40ce-9fb7-058c7a9cd44a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:07.503849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:07.580004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60c302-d868-40ce-9fb7-058c7a9cd44a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:07.625269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47788150-c410-4a13-8211-d923833fd26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:07.625969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47788150-c410-4a13-8211-d923833fd26b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:07.626249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47788150-c410-4a13-8211-d923833fd26b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:07.626564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47788150-c410-4a13-8211-d923833fd26b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:07.627646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47788150-c410-4a13-8211-d923833fd26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:07.629058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47788150-c410-4a13-8211-d923833fd26b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:07.630827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 165/323] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:07 2022] GET /identity/v3/domains/default => generated 200 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:07.636995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f2f19-f41f-43bf-86a7-de6466915d05 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:07.704622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47788150-c410-4a13-8211-d923833fd26b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:07.725933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:07.759754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f2f19-f41f-43bf-86a7-de6466915d05 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:07.799051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3217a-11b8-46b0-bb06-c13b3c97a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:07.799668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3217a-11b8-46b0-bb06-c13b3c97a9da None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:07.799950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3217a-11b8-46b0-bb06-c13b3c97a9da None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:07.800223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3217a-11b8-46b0-bb06-c13b3c97a9da None admin] PATH_INFO: `/v3/groups` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:07.800855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3217a-11b8-46b0-bb06-c13b3c97a9da None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:07.802442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3217a-11b8-46b0-bb06-c13b3c97a9da None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:07.825616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 159/324] 10.222.0.100 () {60 vars in 1139 bytes} [Thu Sep 1 17:23:07 2022] POST /identity/v3/groups => generated 227 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:23:07.841629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47788150-c410-4a13-8211-d923833fd26b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:07.862228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:07.888014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3beb6-1273-4c05-b23e-4b53ccb8a2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:07.888710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3beb6-1273-4c05-b23e-4b53ccb8a2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:07.888989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3beb6-1273-4c05-b23e-4b53ccb8a2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:07.889255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3beb6-1273-4c05-b23e-4b53ccb8a2d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:07.903952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3beb6-1273-4c05-b23e-4b53ccb8a2d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-97b3beb6-1273-4c05-b23e-4b53ccb8a2d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:07.909630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:07.914262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:07.916362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 166/325] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:07.917765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3217a-11b8-46b0-bb06-c13b3c97a9da None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:08.045265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3217a-11b8-46b0-bb06-c13b3c97a9da None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:08.083845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b34eca-6ab5-48be-b9ca-116acea63c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:08.085105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b34eca-6ab5-48be-b9ca-116acea63c71 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:08.085793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b34eca-6ab5-48be-b9ca-116acea63c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:08.085793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b34eca-6ab5-48be-b9ca-116acea63c71 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:08.086364 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b34eca-6ab5-48be-b9ca-116acea63c71 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:08.086663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b34eca-6ab5-48be-b9ca-116acea63c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:08.088057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b34eca-6ab5-48be-b9ca-116acea63c71 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:08.089712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:08.101818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 160/326] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:07 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:08.105224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3beb6-1273-4c05-b23e-4b53ccb8a2d0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:08.234671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3beb6-1273-4c05-b23e-4b53ccb8a2d0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:08.275087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334cee2-4d48-447f-8d00-284e6f0fca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:08.276058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334cee2-4d48-447f-8d00-284e6f0fca32 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:08.276342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334cee2-4d48-447f-8d00-284e6f0fca32 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:08.276616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334cee2-4d48-447f-8d00-284e6f0fca32 None admin] PATH_INFO: `/v3/users/42de788ae47345198b9b6a679a2d2423` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:08.277887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334cee2-4d48-447f-8d00-284e6f0fca32 None admin] RBAC: Authorizing `identity:get_user(user_id=42de788ae47345198b9b6a679a2d2423)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:08.279431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334cee2-4d48-447f-8d00-284e6f0fca32 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:08.281385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 167/327] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:23:08 2022] GET /identity/v3/users/42de788ae47345198b9b6a679a2d2423 => generated 294 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:08.287613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b34eca-6ab5-48be-b9ca-116acea63c71 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:08.348120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c869-818b-4737-a796-5fdfb0f02c11 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:08.348652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c869-818b-4737-a796-5fdfb0f02c11 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:08.349126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c869-818b-4737-a796-5fdfb0f02c11 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:08.352279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 168/328] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:08 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:23:08.359210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f4ba7-4acf-4858-8270-bcf9dd0f06d7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:08.359501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f4ba7-4acf-4858-8270-bcf9dd0f06d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:08.359772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f4ba7-4acf-4858-8270-bcf9dd0f06d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:08.383269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-063f4ba7-4acf-4858-8270-bcf9dd0f06d7 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:08.384889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-063f4ba7-4acf-4858-8270-bcf9dd0f06d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:08.416944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b34eca-6ab5-48be-b9ca-116acea63c71 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:08.438907 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:08.464667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2fe40-36e4-49cb-a2d2-f941118b9208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:08.465326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2fe40-36e4-49cb-a2d2-f941118b9208 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:08.465618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2fe40-36e4-49cb-a2d2-f941118b9208 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:08.465900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2fe40-36e4-49cb-a2d2-f941118b9208 None admin] PATH_INFO: `/v3/system/users/42de788ae47345198b9b6a679a2d2423/roles/e272ecbda33d4b3e9308242d2d4ab302` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:08.467346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2fe40-36e4-49cb-a2d2-f941118b9208 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=42de788ae47345198b9b6a679a2d2423, role_id=e272ecbda33d4b3e9308242d2d4ab302)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:08.468742 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2fe40-36e4-49cb-a2d2-f941118b9208 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:08.479088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 161/329] 10.222.0.100 () {58 vars in 1339 bytes} [Thu Sep 1 17:23:08 2022] PUT /identity/v3/system/users/42de788ae47345198b9b6a679a2d2423/roles/e272ecbda33d4b3e9308242d2d4ab302 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:08.487013 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 169/330] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:23:08.786001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2fe40-36e4-49cb-a2d2-f941118b9208 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:08.863532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:08.923376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2fe40-36e4-49cb-a2d2-f941118b9208 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:08.976008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0fdeb-0c7b-48be-9792-067198bf4d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:08.976657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0fdeb-0c7b-48be-9792-067198bf4d19 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:08.976933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0fdeb-0c7b-48be-9792-067198bf4d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:08.977199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0fdeb-0c7b-48be-9792-067198bf4d19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:08.996267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0fdeb-0c7b-48be-9792-067198bf4d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-64d0fdeb-0c7b-48be-9792-067198bf4d19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:09.001683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:09.002609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:09.003643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 162/331] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:09.011570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f4ba7-4acf-4858-8270-bcf9dd0f06d7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:09.153280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f4ba7-4acf-4858-8270-bcf9dd0f06d7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:09.198441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458ba51-eb0b-4112-8140-d898ba8068b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:09.199215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458ba51-eb0b-4112-8140-d898ba8068b0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:09.199498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458ba51-eb0b-4112-8140-d898ba8068b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:09.199768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458ba51-eb0b-4112-8140-d898ba8068b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:09.200448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458ba51-eb0b-4112-8140-d898ba8068b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:09.200748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458ba51-eb0b-4112-8140-d898ba8068b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:09.202185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458ba51-eb0b-4112-8140-d898ba8068b0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:09.216404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 170/332] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:09 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:09.222995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0fdeb-0c7b-48be-9792-067198bf4d19 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:09.466774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0fdeb-0c7b-48be-9792-067198bf4d19 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:09.512803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:09.529948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0ed41-dd7f-4f00-801a-9eaa313cac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:09.530607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0ed41-dd7f-4f00-801a-9eaa313cac3f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:09.530886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0ed41-dd7f-4f00-801a-9eaa313cac3f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:09.531159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0ed41-dd7f-4f00-801a-9eaa313cac3f None admin] PATH_INFO: `/v3/users/b162f00c4c5b40e7858f82dce0708608` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:09.532409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0ed41-dd7f-4f00-801a-9eaa313cac3f None admin] RBAC: Authorizing `identity:get_user(user_id=b162f00c4c5b40e7858f82dce0708608)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:09.533933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0ed41-dd7f-4f00-801a-9eaa313cac3f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:09.535953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 163/333] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:23:09 2022] GET /identity/v3/users/b162f00c4c5b40e7858f82dce0708608 => generated 294 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:09.543003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458ba51-eb0b-4112-8140-d898ba8068b0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:09.605436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:09.693362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458ba51-eb0b-4112-8140-d898ba8068b0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:09.733811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da2247-10e3-431e-9e85-690166a898b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:09.734328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da2247-10e3-431e-9e85-690166a898b2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:09.734615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da2247-10e3-431e-9e85-690166a898b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:09.734868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da2247-10e3-431e-9e85-690166a898b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:09.735598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da2247-10e3-431e-9e85-690166a898b2 None admin] RBAC: Adding query filter params (role.id=90949fa670c2407ca92b16156e52f47f, user.id=b162f00c4c5b40e7858f82dce0708608, scope.system=all) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:09.735899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da2247-10e3-431e-9e85-690166a898b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:09.737345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da2247-10e3-431e-9e85-690166a898b2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:09.751293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 171/334] 10.222.0.100 () {56 vars in 1313 bytes} [Thu Sep 1 17:23:09 2022] GET /identity/v3/role_assignments?user.id=b162f00c4c5b40e7858f82dce0708608&scope.system=all&role.id=90949fa670c2407ca92b16156e52f47f => generated 512 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:10.649173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bacdb-aa81-4455-8abc-0a07d0a27bcf None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:10.649747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bacdb-aa81-4455-8abc-0a07d0a27bcf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:10.649747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bacdb-aa81-4455-8abc-0a07d0a27bcf None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:10.651601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 164/335] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:23:10.668954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47198e-2482-405a-93f0-9b1553c3b31c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:10.669513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47198e-2482-405a-93f0-9b1553c3b31c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:10.669951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47198e-2482-405a-93f0-9b1553c3b31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:10.703559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ef47198e-2482-405a-93f0-9b1553c3b31c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:10.704753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ef47198e-2482-405a-93f0-9b1553c3b31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:10.788736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 172/336] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:11.033138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f6a7-c81a-41a2-9e81-9e40d14dfa3f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:11.033439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f6a7-c81a-41a2-9e81-9e40d14dfa3f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:11.033647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f6a7-c81a-41a2-9e81-9e40d14dfa3f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:11.035020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 165/337] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:11.047360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c35a-cb9a-4f2c-9c8b-7567207e31a8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:11.047668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c35a-cb9a-4f2c-9c8b-7567207e31a8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:11.047909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c35a-cb9a-4f2c-9c8b-7567207e31a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:11.068366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9122c35a-cb9a-4f2c-9c8b-7567207e31a8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:11.070482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9122c35a-cb9a-4f2c-9c8b-7567207e31a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:11.094789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:11.160808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 173/338] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:11.190961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f498d28-ebfb-4019-9af2-57ac88cd641e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:11.191927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f498d28-ebfb-4019-9af2-57ac88cd641e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:11.192687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f498d28-ebfb-4019-9af2-57ac88cd641e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:11.196549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 166/339] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:11 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:11.205132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c80fe-425d-4ad3-b73c-b61b3deaa1d1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:11.205492 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c80fe-425d-4ad3-b73c-b61b3deaa1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:11.205778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c80fe-425d-4ad3-b73c-b61b3deaa1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:11.218252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f498d28-ebfb-4019-9af2-57ac88cd641e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:11.228048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2e5c80fe-425d-4ad3-b73c-b61b3deaa1d1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:11.229624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2e5c80fe-425d-4ad3-b73c-b61b3deaa1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:11.359400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 174/340] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:11.405138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f498d28-ebfb-4019-9af2-57ac88cd641e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:11.450109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07de58-7327-48be-8f99-f065dc7646d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:11.451217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07de58-7327-48be-8f99-f065dc7646d0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:11.451715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07de58-7327-48be-8f99-f065dc7646d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:11.452200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07de58-7327-48be-8f99-f065dc7646d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:11.465670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07de58-7327-48be-8f99-f065dc7646d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9b07de58-7327-48be-8f99-f065dc7646d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:11.471165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:11.472107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:11.472916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:11.476202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 167/341] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:11.481985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c80fe-425d-4ad3-b73c-b61b3deaa1d1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:11.552559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07de58-7327-48be-8f99-f065dc7646d0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:11.601889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:11.636985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:11.643312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c80fe-425d-4ad3-b73c-b61b3deaa1d1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:11.730059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6302d-c930-47c9-8937-0efc382a135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:11.730715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6302d-c930-47c9-8937-0efc382a135a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:11.730983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6302d-c930-47c9-8937-0efc382a135a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:11.731292 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6302d-c930-47c9-8937-0efc382a135a None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:11.733468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6302d-c930-47c9-8937-0efc382a135a None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:11.733468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6302d-c930-47c9-8937-0efc382a135a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:11.733672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6302d-c930-47c9-8937-0efc382a135a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:11.752095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 175/342] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:11 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:11.754043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07de58-7327-48be-8f99-f065dc7646d0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:11.759190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6302d-c930-47c9-8937-0efc382a135a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:11.819960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd4e03-1f54-4b8d-9f40-6ccbe300a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:11.820998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd4e03-1f54-4b8d-9f40-6ccbe300a53f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:11.821488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd4e03-1f54-4b8d-9f40-6ccbe300a53f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:11.821995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd4e03-1f54-4b8d-9f40-6ccbe300a53f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:11.842726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd4e03-1f54-4b8d-9f40-6ccbe300a53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-47dd4e03-1f54-4b8d-9f40-6ccbe300a53f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:11.850189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:11.851151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:11.851932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:11.859007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 168/343] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:23:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:11.866429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd4e03-1f54-4b8d-9f40-6ccbe300a53f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:11.967389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6302d-c930-47c9-8937-0efc382a135a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:12.021137 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1d3a-770a-49eb-9de1-45579d346d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:12.022220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1d3a-770a-49eb-9de1-45579d346d52 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.023670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1d3a-770a-49eb-9de1-45579d346d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.024198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1d3a-770a-49eb-9de1-45579d346d52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.045543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:12.105613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd4e03-1f54-4b8d-9f40-6ccbe300a53f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:12.124000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:12.139789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1d3a-770a-49eb-9de1-45579d346d52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:12.143754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1d3a-770a-49eb-9de1-45579d346d52 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:12.188514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fae886-27f5-40fd-9758-e77d8e5169bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:12.189384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fae886-27f5-40fd-9758-e77d8e5169bd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.190082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fae886-27f5-40fd-9758-e77d8e5169bd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.191131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fae886-27f5-40fd-9758-e77d8e5169bd None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.192060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fae886-27f5-40fd-9758-e77d8e5169bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:12.192379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fae886-27f5-40fd-9758-e77d8e5169bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:12.195317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fae886-27f5-40fd-9758-e77d8e5169bd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:12.208098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 169/344] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:11 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:12.209266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fae886-27f5-40fd-9758-e77d8e5169bd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:12.285809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 176/345] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:11 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:12.287249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824f93c-24f2-4714-af1c-558a3763a192 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.287564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824f93c-24f2-4714-af1c-558a3763a192 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.287844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824f93c-24f2-4714-af1c-558a3763a192 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.289805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 177/346] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:12.290596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824f93c-24f2-4714-af1c-558a3763a192 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:12.351262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fae886-27f5-40fd-9758-e77d8e5169bd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:12.392240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:12.392931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872963d-ac0e-479b-8e60-9fa0d017d0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:12.393699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872963d-ac0e-479b-8e60-9fa0d017d0dc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.394289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872963d-ac0e-479b-8e60-9fa0d017d0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.394618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872963d-ac0e-479b-8e60-9fa0d017d0dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.411178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872963d-ac0e-479b-8e60-9fa0d017d0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d872963d-ac0e-479b-8e60-9fa0d017d0dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:12.418530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:12.419859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:12.420767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 170/347] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:12.421558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665c3d1-9d22-4d92-8b9c-e5453875b4e9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.421558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665c3d1-9d22-4d92-8b9c-e5453875b4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.421558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665c3d1-9d22-4d92-8b9c-e5453875b4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.460292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6665c3d1-9d22-4d92-8b9c-e5453875b4e9 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:12.461955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6665c3d1-9d22-4d92-8b9c-e5453875b4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:12.493045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824f93c-24f2-4714-af1c-558a3763a192 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:12.495708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:12.549220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc1286-f636-40c1-a710-5d8c56870f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:12.549220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc1286-f636-40c1-a710-5d8c56870f85 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.549220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc1286-f636-40c1-a710-5d8c56870f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.549220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc1286-f636-40c1-a710-5d8c56870f85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.575086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 171/348] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:12.576278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665c3d1-9d22-4d92-8b9c-e5453875b4e9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:12.640547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc1286-f636-40c1-a710-5d8c56870f85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:12.642077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc1286-f636-40c1-a710-5d8c56870f85 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:12.713417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665c3d1-9d22-4d92-8b9c-e5453875b4e9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:12.734064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:12.766962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19c0b1-18f6-4f8a-b597-17ed58fa9848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:12.767766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19c0b1-18f6-4f8a-b597-17ed58fa9848 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.768882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19c0b1-18f6-4f8a-b597-17ed58fa9848 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.768882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19c0b1-18f6-4f8a-b597-17ed58fa9848 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.770477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19c0b1-18f6-4f8a-b597-17ed58fa9848 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:12.773625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19c0b1-18f6-4f8a-b597-17ed58fa9848 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:12.775993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 172/349] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:12 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:12.777314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19c0b1-18f6-4f8a-b597-17ed58fa9848 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:12.780525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 178/350] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:12 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:12.782230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f0e62-6ff9-456d-b0d7-89ca34fbef00 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.782699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f0e62-6ff9-456d-b0d7-89ca34fbef00 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.783115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f0e62-6ff9-456d-b0d7-89ca34fbef00 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.789195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 179/351] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:12.789195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f0e62-6ff9-456d-b0d7-89ca34fbef00 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:12.826766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:12.920848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f0e62-6ff9-456d-b0d7-89ca34fbef00 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:12.923276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19c0b1-18f6-4f8a-b597-17ed58fa9848 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:12.964194 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb85fa9-a4d2-4099-a53a-224f57d2b25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:12.965676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb85fa9-a4d2-4099-a53a-224f57d2b25e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.965676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb85fa9-a4d2-4099-a53a-224f57d2b25e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.966039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb85fa9-a4d2-4099-a53a-224f57d2b25e None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.972569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8b24f-38d3-46ed-ba78-fc2b6a3393ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:12.972569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8b24f-38d3-46ed-ba78-fc2b6a3393ec None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.972569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8b24f-38d3-46ed-ba78-fc2b6a3393ec None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.972569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8b24f-38d3-46ed-ba78-fc2b6a3393ec None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:12.972938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8b24f-38d3-46ed-ba78-fc2b6a3393ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:12.972938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8b24f-38d3-46ed-ba78-fc2b6a3393ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:12.976729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb85fa9-a4d2-4099-a53a-224f57d2b25e None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:12.976881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8b24f-38d3-46ed-ba78-fc2b6a3393ec None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:12.976959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb85fa9-a4d2-4099-a53a-224f57d2b25e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:12.977936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 180/352] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:12 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:12.979694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb85fa9-a4d2-4099-a53a-224f57d2b25e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:12.987470 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 173/353] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:12 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:12.990050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bb59b-e6d0-4244-bea8-5362b55239dc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:12.990656 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bb59b-e6d0-4244-bea8-5362b55239dc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:12.992051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:12.992626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bb59b-e6d0-4244-bea8-5362b55239dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.013414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3a1bb59b-e6d0-4244-bea8-5362b55239dc None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:13.015018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3a1bb59b-e6d0-4244-bea8-5362b55239dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:13.119381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 174/354] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:13.120169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb85fa9-a4d2-4099-a53a-224f57d2b25e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:13.120300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bb59b-e6d0-4244-bea8-5362b55239dc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:13.152837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:13.169261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ce923-a4fc-45ba-9d2e-113d76ea77f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:13.169929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ce923-a4fc-45ba-9d2e-113d76ea77f5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:13.170205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ce923-a4fc-45ba-9d2e-113d76ea77f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:13.170469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ce923-a4fc-45ba-9d2e-113d76ea77f5 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.171604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ce923-a4fc-45ba-9d2e-113d76ea77f5 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:13.173224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ce923-a4fc-45ba-9d2e-113d76ea77f5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:13.175439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 181/355] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:12 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:13.176700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ce923-a4fc-45ba-9d2e-113d76ea77f5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:13.258399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bb59b-e6d0-4244-bea8-5362b55239dc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:13.277734 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:13.311652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ce923-a4fc-45ba-9d2e-113d76ea77f5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:13.315405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbddab6-bca6-440e-b3e8-fed2e6e706e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:13.316106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbddab6-bca6-440e-b3e8-fed2e6e706e9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:13.316387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbddab6-bca6-440e-b3e8-fed2e6e706e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:13.316654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbddab6-bca6-440e-b3e8-fed2e6e706e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.331484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbddab6-bca6-440e-b3e8-fed2e6e706e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-adbddab6-bca6-440e-b3e8-fed2e6e706e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:13.338730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:13.339675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:13.340478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 175/356] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:13.341150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbddab6-bca6-440e-b3e8-fed2e6e706e9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:13.365618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6af22-44cf-4347-a538-4c6c0e49aab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:13.367279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6af22-44cf-4347-a538-4c6c0e49aab3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:13.367625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6af22-44cf-4347-a538-4c6c0e49aab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:13.367904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6af22-44cf-4347-a538-4c6c0e49aab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.368679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6af22-44cf-4347-a538-4c6c0e49aab3 None admin] RBAC: Adding query filter params (group.id=0651fc7bb71542c287b073460128eef5, role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=6fc37cae8bab49beb9244a40222ffda1) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:13.368946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6af22-44cf-4347-a538-4c6c0e49aab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:13.370600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6af22-44cf-4347-a538-4c6c0e49aab3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:13.377868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 182/357] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:13 2022] GET /identity/v3/role_assignments?group.id=0651fc7bb71542c287b073460128eef5&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=90949fa670c2407ca92b16156e52f47f => generated 261 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:13.378732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6af22-44cf-4347-a538-4c6c0e49aab3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:13.455050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:13.483235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbddab6-bca6-440e-b3e8-fed2e6e706e9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:13.534624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf5a9a-b37d-42e3-ba5a-223322910c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:13.535904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf5a9a-b37d-42e3-ba5a-223322910c5b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:13.536855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf5a9a-b37d-42e3-ba5a-223322910c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:13.537340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf5a9a-b37d-42e3-ba5a-223322910c5b None admin] PATH_INFO: `/v3/users/42de788ae47345198b9b6a679a2d2423` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.539484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf5a9a-b37d-42e3-ba5a-223322910c5b None admin] RBAC: Authorizing `identity:get_user(user_id=42de788ae47345198b9b6a679a2d2423)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:13.541202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf5a9a-b37d-42e3-ba5a-223322910c5b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:13.543939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 176/358] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:23:13 2022] GET /identity/v3/users/42de788ae47345198b9b6a679a2d2423 => generated 294 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:13.545460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1cd8a-f5df-4af7-8bad-2d8279e69f27 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:13.546064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1cd8a-f5df-4af7-8bad-2d8279e69f27 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:13.546603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1cd8a-f5df-4af7-8bad-2d8279e69f27 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.548333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6af22-44cf-4347-a538-4c6c0e49aab3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:13.548505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 177/359] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:13.549895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1cd8a-f5df-4af7-8bad-2d8279e69f27 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:13.607169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5ffe-7330-4d0a-8126-fec29ba66698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:13.607833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5ffe-7330-4d0a-8126-fec29ba66698 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:13.608121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5ffe-7330-4d0a-8126-fec29ba66698 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:13.608393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5ffe-7330-4d0a-8126-fec29ba66698 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.610490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd5ffe-7330-4d0a-8126-fec29ba66698 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:13.612455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd5ffe-7330-4d0a-8126-fec29ba66698 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:13.615300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 183/360] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:13 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:13.620932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5ffe-7330-4d0a-8126-fec29ba66698 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:13.633556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:13.730535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1cd8a-f5df-4af7-8bad-2d8279e69f27 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:13.759132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5ffe-7330-4d0a-8126-fec29ba66698 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:13.784539 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054adf5-ef8e-4ca0-908f-0ed7e23b47b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:13.785185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054adf5-ef8e-4ca0-908f-0ed7e23b47b1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:13.785436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054adf5-ef8e-4ca0-908f-0ed7e23b47b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:13.785744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054adf5-ef8e-4ca0-908f-0ed7e23b47b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.786430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054adf5-ef8e-4ca0-908f-0ed7e23b47b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:13.786738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054adf5-ef8e-4ca0-908f-0ed7e23b47b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:13.788818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054adf5-ef8e-4ca0-908f-0ed7e23b47b1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:13.799826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 178/361] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:13 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:13.799826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054adf5-ef8e-4ca0-908f-0ed7e23b47b1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:13.813725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a2dca-0a12-4f6a-bec3-8378224bc198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:13.814562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a2dca-0a12-4f6a-bec3-8378224bc198 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:13.814910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a2dca-0a12-4f6a-bec3-8378224bc198 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:13.815202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a2dca-0a12-4f6a-bec3-8378224bc198 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.815811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:13.841837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a2dca-0a12-4f6a-bec3-8378224bc198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-629a2dca-0a12-4f6a-bec3-8378224bc198 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:13.843680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:13.844631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:13.845433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:13.849690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 184/362] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:23:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:13.852594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945f7f-3e6e-4738-b4d2-5f6646b7b35e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:13.852889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945f7f-3e6e-4738-b4d2-5f6646b7b35e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:13.853161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945f7f-3e6e-4738-b4d2-5f6646b7b35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:13.876401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a2945f7f-3e6e-4738-b4d2-5f6646b7b35e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:13.877653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a2945f7f-3e6e-4738-b4d2-5f6646b7b35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:13.955493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054adf5-ef8e-4ca0-908f-0ed7e23b47b1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:13.960198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:13.967923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 185/363] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:13.968817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945f7f-3e6e-4738-b4d2-5f6646b7b35e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:14.001448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b943-2786-4a9f-87d3-04cd9de32305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:14.002501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b943-2786-4a9f-87d3-04cd9de32305 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:14.003023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b943-2786-4a9f-87d3-04cd9de32305 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:14.003532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b943-2786-4a9f-87d3-04cd9de32305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:14.004800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b943-2786-4a9f-87d3-04cd9de32305 None admin] RBAC: Adding query filter params (role.id=e272ecbda33d4b3e9308242d2d4ab302, user.id=42de788ae47345198b9b6a679a2d2423, scope.system=all) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:14.005534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b943-2786-4a9f-87d3-04cd9de32305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:14.008793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b943-2786-4a9f-87d3-04cd9de32305 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:14.029911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 179/364] 10.222.0.100 () {56 vars in 1313 bytes} [Thu Sep 1 17:23:13 2022] GET /identity/v3/role_assignments?user.id=42de788ae47345198b9b6a679a2d2423&scope.system=all&role.id=e272ecbda33d4b3e9308242d2d4ab302 => generated 512 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:14.031072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b943-2786-4a9f-87d3-04cd9de32305 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:14.154532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945f7f-3e6e-4738-b4d2-5f6646b7b35e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:14.164846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:14.222751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2300c8-e0e1-41c8-a9b5-bf76644bb1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:14.223694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2300c8-e0e1-41c8-a9b5-bf76644bb1ef None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:14.224179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2300c8-e0e1-41c8-a9b5-bf76644bb1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:14.224635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2300c8-e0e1-41c8-a9b5-bf76644bb1ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:14.226220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2300c8-e0e1-41c8-a9b5-bf76644bb1ef None admin] RBAC: Adding query filter params (group.id=0651fc7bb71542c287b073460128eef5, role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=6fc37cae8bab49beb9244a40222ffda1) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:14.226919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2300c8-e0e1-41c8-a9b5-bf76644bb1ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:14.228885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2300c8-e0e1-41c8-a9b5-bf76644bb1ef None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:14.237922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 186/365] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:13 2022] GET /identity/v3/role_assignments?group.id=0651fc7bb71542c287b073460128eef5&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=5dc7056ddc154b81939a17474a64f546 => generated 261 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:14.239566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2300c8-e0e1-41c8-a9b5-bf76644bb1ef None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:14.244061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b943-2786-4a9f-87d3-04cd9de32305 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:14.298151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3e048-48f2-4eef-93d9-97f3d7528d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:14.298772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3e048-48f2-4eef-93d9-97f3d7528d37 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:14.299051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3e048-48f2-4eef-93d9-97f3d7528d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:14.299332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3e048-48f2-4eef-93d9-97f3d7528d37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:14.403005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:14.448815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3e048-48f2-4eef-93d9-97f3d7528d37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:14.450449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3e048-48f2-4eef-93d9-97f3d7528d37 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:14.458656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2300c8-e0e1-41c8-a9b5-bf76644bb1ef None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:14.517327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc178f3-cc27-4c2c-96d1-6f98f165fc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:14.518753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc178f3-cc27-4c2c-96d1-6f98f165fc84 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:14.519278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc178f3-cc27-4c2c-96d1-6f98f165fc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:14.519846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc178f3-cc27-4c2c-96d1-6f98f165fc84 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:14.521129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc178f3-cc27-4c2c-96d1-6f98f165fc84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:14.522191 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc178f3-cc27-4c2c-96d1-6f98f165fc84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:14.524013 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc178f3-cc27-4c2c-96d1-6f98f165fc84 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:14.542089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 187/366] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:14 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:14.543658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc178f3-cc27-4c2c-96d1-6f98f165fc84 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:14.554353 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:14.615563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 180/367] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:14.616588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3e048-48f2-4eef-93d9-97f3d7528d37 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:14.735813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc178f3-cc27-4c2c-96d1-6f98f165fc84 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:14.765178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3e048-48f2-4eef-93d9-97f3d7528d37 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:14.768894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:14.782407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadae861-2ee2-4ed1-8ae9-7841dbf43e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:14.783578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadae861-2ee2-4ed1-8ae9-7841dbf43e57 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:14.784005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadae861-2ee2-4ed1-8ae9-7841dbf43e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:14.784279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadae861-2ee2-4ed1-8ae9-7841dbf43e57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:14.799293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadae861-2ee2-4ed1-8ae9-7841dbf43e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-cadae861-2ee2-4ed1-8ae9-7841dbf43e57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:14.806993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:14.807984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:23:14.808777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:14.809413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 188/368] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:23:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:14.811318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadae861-2ee2-4ed1-8ae9-7841dbf43e57 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:14.820022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7b51e-880f-4a16-b9ec-f2970f06aaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:14.820664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7b51e-880f-4a16-b9ec-f2970f06aaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:14.820917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7b51e-880f-4a16-b9ec-f2970f06aaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:14.821187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7b51e-880f-4a16-b9ec-f2970f06aaf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:14.908366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:14.959902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7b51e-880f-4a16-b9ec-f2970f06aaf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:14.962213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7b51e-880f-4a16-b9ec-f2970f06aaf8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:14.977126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadae861-2ee2-4ed1-8ae9-7841dbf43e57 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:15.016301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51d512-2984-4ba1-83cf-c5d3c6971c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:15.016839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51d512-2984-4ba1-83cf-c5d3c6971c86 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.017127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51d512-2984-4ba1-83cf-c5d3c6971c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.017402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51d512-2984-4ba1-83cf-c5d3c6971c86 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.018616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51d512-2984-4ba1-83cf-c5d3c6971c86 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:15.020103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51d512-2984-4ba1-83cf-c5d3c6971c86 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:15.022719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 189/369] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:14 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:15.023543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51d512-2984-4ba1-83cf-c5d3c6971c86 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:15.084237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:15.105785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 181/370] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:15.106787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7b51e-880f-4a16-b9ec-f2970f06aaf8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:15.139335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51d512-2984-4ba1-83cf-c5d3c6971c86 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:15.178649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e5111-5943-4cc1-928e-50f0cb277f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:15.179277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e5111-5943-4cc1-928e-50f0cb277f64 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.179557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e5111-5943-4cc1-928e-50f0cb277f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.179833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e5111-5943-4cc1-928e-50f0cb277f64 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.180519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e5111-5943-4cc1-928e-50f0cb277f64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:15.180824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e5111-5943-4cc1-928e-50f0cb277f64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:15.182278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e5111-5943-4cc1-928e-50f0cb277f64 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:15.187135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:15.192285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 190/371] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:15 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:15.193165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e5111-5943-4cc1-928e-50f0cb277f64 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:15.230590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7b51e-880f-4a16-b9ec-f2970f06aaf8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:15.272564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb97c3-4a88-440a-b639-88adb511cf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:15.273445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb97c3-4a88-440a-b639-88adb511cf28 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.273804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb97c3-4a88-440a-b639-88adb511cf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.274184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb97c3-4a88-440a-b639-88adb511cf28 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.275345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb97c3-4a88-440a-b639-88adb511cf28 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:15.277063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb97c3-4a88-440a-b639-88adb511cf28 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:15.284121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 182/372] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:15 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:15.285077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb97c3-4a88-440a-b639-88adb511cf28 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:15.311583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e5111-5943-4cc1-928e-50f0cb277f64 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:15.326946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:15.349328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a262d-0f39-425b-9649-73db6c4e9a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:15.350172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a262d-0f39-425b-9649-73db6c4e9a3b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.350528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a262d-0f39-425b-9649-73db6c4e9a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.350878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a262d-0f39-425b-9649-73db6c4e9a3b None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.352166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a262d-0f39-425b-9649-73db6c4e9a3b None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:15.353714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a262d-0f39-425b-9649-73db6c4e9a3b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:15.355618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 191/373] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:15 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:15.356230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a262d-0f39-425b-9649-73db6c4e9a3b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:15.405803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb97c3-4a88-440a-b639-88adb511cf28 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:15.447226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e6a8c-7a57-43ec-96bd-590779ef837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:15.447847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e6a8c-7a57-43ec-96bd-590779ef837a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.448125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e6a8c-7a57-43ec-96bd-590779ef837a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.448499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e6a8c-7a57-43ec-96bd-590779ef837a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.452641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:15.471400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a262d-0f39-425b-9649-73db6c4e9a3b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:15.512970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75588d3-9f53-4c11-b160-cea0de90a3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:15.513612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75588d3-9f53-4c11-b160-cea0de90a3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.513903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75588d3-9f53-4c11-b160-cea0de90a3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.514186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75588d3-9f53-4c11-b160-cea0de90a3c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.514923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75588d3-9f53-4c11-b160-cea0de90a3c5 None admin] RBAC: Adding query filter params (group.id=0651fc7bb71542c287b073460128eef5, role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=4d0f6555111843d2af0acf5759071078) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:15.515234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75588d3-9f53-4c11-b160-cea0de90a3c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:15.516688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75588d3-9f53-4c11-b160-cea0de90a3c5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:15.523475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 192/374] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:15 2022] GET /identity/v3/role_assignments?group.id=0651fc7bb71542c287b073460128eef5&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=90949fa670c2407ca92b16156e52f47f => generated 261 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:15.524205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75588d3-9f53-4c11-b160-cea0de90a3c5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:15.535737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e6a8c-7a57-43ec-96bd-590779ef837a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:15.537393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e6a8c-7a57-43ec-96bd-590779ef837a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:15.577380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:15.648372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75588d3-9f53-4c11-b160-cea0de90a3c5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:15.675438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 183/375] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:15 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:15.677020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e6a8c-7a57-43ec-96bd-590779ef837a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:15.689408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26188f93-95e9-4eea-9387-7b78081cf4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:15.690173 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26188f93-95e9-4eea-9387-7b78081cf4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.690459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26188f93-95e9-4eea-9387-7b78081cf4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.690733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26188f93-95e9-4eea-9387-7b78081cf4a2 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.691905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26188f93-95e9-4eea-9387-7b78081cf4a2 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:15.693364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26188f93-95e9-4eea-9387-7b78081cf4a2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:15.695494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 193/376] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:15 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:15.702650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26188f93-95e9-4eea-9387-7b78081cf4a2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:15.721279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:15.806616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e6a8c-7a57-43ec-96bd-590779ef837a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:15.857869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b3217-0745-4833-bc91-01afd5a19adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:15.858989 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b3217-0745-4833-bc91-01afd5a19adc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.859906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b3217-0745-4833-bc91-01afd5a19adc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.860401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b3217-0745-4833-bc91-01afd5a19adc None admin] PATH_INFO: `/v3/projects/392e00333a6b41639a27c8ea706c3db5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.863431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b3217-0745-4833-bc91-01afd5a19adc None admin] RBAC: Authorizing `identity:get_project(project_id=392e00333a6b41639a27c8ea706c3db5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:15.866409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b3217-0745-4833-bc91-01afd5a19adc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:15.869423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26188f93-95e9-4eea-9387-7b78081cf4a2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:15.870132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 184/377] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:15 2022] GET /identity/v3/projects/392e00333a6b41639a27c8ea706c3db5 => generated 345 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:15.879129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077f44b-03c1-4477-8422-661c2a381427 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.879630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077f44b-03c1-4477-8422-661c2a381427 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.879936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077f44b-03c1-4477-8422-661c2a381427 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.881371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 185/378] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:15 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:15.883819 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077f44b-03c1-4477-8422-661c2a381427 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:15.908477 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:15.913281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001855-85ef-4992-870a-3697c40974cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:15.913944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001855-85ef-4992-870a-3697c40974cb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.914234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001855-85ef-4992-870a-3697c40974cb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.914517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001855-85ef-4992-870a-3697c40974cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.915374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001855-85ef-4992-870a-3697c40974cb None admin] RBAC: Adding query filter params (group.id=0651fc7bb71542c287b073460128eef5, role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=4d0f6555111843d2af0acf5759071078) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:15.915711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001855-85ef-4992-870a-3697c40974cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:15.917254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001855-85ef-4992-870a-3697c40974cb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:15.925053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 194/379] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:15 2022] GET /identity/v3/role_assignments?group.id=0651fc7bb71542c287b073460128eef5&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=5dc7056ddc154b81939a17474a64f546 => generated 261 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:15.926936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777dd83-ec3c-4d0c-b28e-bf285c1952b7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:15.927270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777dd83-ec3c-4d0c-b28e-bf285c1952b7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:15.927644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777dd83-ec3c-4d0c-b28e-bf285c1952b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:15.950709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6777dd83-ec3c-4d0c-b28e-bf285c1952b7 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:15.952187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6777dd83-ec3c-4d0c-b28e-bf285c1952b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:16.013917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077f44b-03c1-4477-8422-661c2a381427 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:16.016898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:16.057676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 195/380] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:16.077447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621b3da-8b17-4d41-8407-e6b08e24424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:16.078180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621b3da-8b17-4d41-8407-e6b08e24424c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:16.078491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621b3da-8b17-4d41-8407-e6b08e24424c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:16.078760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621b3da-8b17-4d41-8407-e6b08e24424c None admin] PATH_INFO: `/v3/groups/bf6fe40707c04f3abf1ca19d39e10e2c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:16.090949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621b3da-8b17-4d41-8407-e6b08e24424c None admin] RBAC: Authorizing `identity:get_group(group_id=bf6fe40707c04f3abf1ca19d39e10e2c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:16.092457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621b3da-8b17-4d41-8407-e6b08e24424c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:16.094576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 186/381] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:15 2022] GET /identity/v3/groups/bf6fe40707c04f3abf1ca19d39e10e2c => generated 230 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:16.100346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777dd83-ec3c-4d0c-b28e-bf285c1952b7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:16.234223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777dd83-ec3c-4d0c-b28e-bf285c1952b7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:16.246773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:16.277568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3eb14-859b-4fee-bbed-b37c5fed57ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:16.278368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3eb14-859b-4fee-bbed-b37c5fed57ba None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:16.278659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3eb14-859b-4fee-bbed-b37c5fed57ba None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:16.278956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3eb14-859b-4fee-bbed-b37c5fed57ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:16.279740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3eb14-859b-4fee-bbed-b37c5fed57ba None admin] RBAC: Adding query filter params (group.id=bf6fe40707c04f3abf1ca19d39e10e2c, role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=392e00333a6b41639a27c8ea706c3db5) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:16.280075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3eb14-859b-4fee-bbed-b37c5fed57ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:16.281692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3eb14-859b-4fee-bbed-b37c5fed57ba None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:16.289818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 196/382] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:16 2022] GET /identity/v3/role_assignments?group.id=bf6fe40707c04f3abf1ca19d39e10e2c&scope.project.id=392e00333a6b41639a27c8ea706c3db5&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 261 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:16.368335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621b3da-8b17-4d41-8407-e6b08e24424c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:16.499092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621b3da-8b17-4d41-8407-e6b08e24424c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:16.541438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3ba4f-73d7-4c0c-bd84-40eac7dd5b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:16.542160 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3ba4f-73d7-4c0c-bd84-40eac7dd5b42 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:16.542440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3ba4f-73d7-4c0c-bd84-40eac7dd5b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:16.542714 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3ba4f-73d7-4c0c-bd84-40eac7dd5b42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:16.557117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3ba4f-73d7-4c0c-bd84-40eac7dd5b42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f0d3ba4f-73d7-4c0c-bd84-40eac7dd5b42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:16.562722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:16.564868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:16.566882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 187/383] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:16.574947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3eb14-859b-4fee-bbed-b37c5fed57ba None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:16.727428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3eb14-859b-4fee-bbed-b37c5fed57ba None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:16.776577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47bcc9-6ce3-45cc-9533-d12ccdf5e5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:16.777209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47bcc9-6ce3-45cc-9533-d12ccdf5e5fe None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:16.777491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47bcc9-6ce3-45cc-9533-d12ccdf5e5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:16.777771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47bcc9-6ce3-45cc-9533-d12ccdf5e5fe None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:16.778462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47bcc9-6ce3-45cc-9533-d12ccdf5e5fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:16.778770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47bcc9-6ce3-45cc-9533-d12ccdf5e5fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:16.780159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47bcc9-6ce3-45cc-9533-d12ccdf5e5fe None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:16.794446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 197/384] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:16 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:16.800537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3ba4f-73d7-4c0c-bd84-40eac7dd5b42 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:16.846804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:16.950245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3ba4f-73d7-4c0c-bd84-40eac7dd5b42 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:16.957387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01b72b-993f-4bed-82e1-8200a3b20a9e None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:16.957906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01b72b-993f-4bed-82e1-8200a3b20a9e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:16.958105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01b72b-993f-4bed-82e1-8200a3b20a9e None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:16.962773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 198/385] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:16.971027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c23957-e665-4020-92f6-72cbf07138bd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:16.971639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c23957-e665-4020-92f6-72cbf07138bd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:16.972107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c23957-e665-4020-92f6-72cbf07138bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:16.994672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca318-064c-4616-abf7-30564ee9e2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:16.995326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca318-064c-4616-abf7-30564ee9e2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:16.995608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca318-064c-4616-abf7-30564ee9e2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:16.995882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca318-064c-4616-abf7-30564ee9e2b1 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:16.997031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cca318-064c-4616-abf7-30564ee9e2b1 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:17.001528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cca318-064c-4616-abf7-30564ee9e2b1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:17.001528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 188/386] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:16 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:17.009646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca318-064c-4616-abf7-30564ee9e2b1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:17.014144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f9c23957-e665-4020-92f6-72cbf07138bd None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:17.015676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9c23957-e665-4020-92f6-72cbf07138bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:17.102666 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:17.124192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca318-064c-4616-abf7-30564ee9e2b1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:17.138017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:17.164782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcde200-2025-459f-9762-19d35e10a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:17.165787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcde200-2025-459f-9762-19d35e10a678 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:17.166235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcde200-2025-459f-9762-19d35e10a678 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:17.166676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcde200-2025-459f-9762-19d35e10a678 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:17.168099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcde200-2025-459f-9762-19d35e10a678 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:17.169786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcde200-2025-459f-9762-19d35e10a678 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:17.172122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 189/387] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:17 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:17.178261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcde200-2025-459f-9762-19d35e10a678 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:17.205895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 199/388] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:17.289668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcde200-2025-459f-9762-19d35e10a678 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:17.325375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfbe17-087c-484e-91e8-744597fcfea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:17.326404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfbe17-087c-484e-91e8-744597fcfea6 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:17.326866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfbe17-087c-484e-91e8-744597fcfea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:17.327322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfbe17-087c-484e-91e8-744597fcfea6 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1/groups/0651fc7bb71542c287b073460128eef5/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:17.329550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfbe17-087c-484e-91e8-744597fcfea6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc37cae8bab49beb9244a40222ffda1, group_id=0651fc7bb71542c287b073460128eef5, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:17.331234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfbe17-087c-484e-91e8-744597fcfea6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:17.349371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 190/389] 10.222.0.100 () {58 vars in 1447 bytes} [Thu Sep 1 17:23:17 2022] PUT /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1/groups/0651fc7bb71542c287b073460128eef5/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:17.666716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c23957-e665-4020-92f6-72cbf07138bd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:17.808535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c23957-e665-4020-92f6-72cbf07138bd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:17.849404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33677cd-c9f8-47d8-b6da-2310dd0a2170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:17.850706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33677cd-c9f8-47d8-b6da-2310dd0a2170 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:17.851031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33677cd-c9f8-47d8-b6da-2310dd0a2170 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:17.851307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33677cd-c9f8-47d8-b6da-2310dd0a2170 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:17.864387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33677cd-c9f8-47d8-b6da-2310dd0a2170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b33677cd-c9f8-47d8-b6da-2310dd0a2170 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:17.870104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:17.871017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:17.871814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 200/390] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:23:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:17.877605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfbe17-087c-484e-91e8-744597fcfea6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:17.928265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:18.014516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfbe17-087c-484e-91e8-744597fcfea6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:18.052481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4af30-fcb9-469d-abba-aecf3cb7046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:18.053655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4af30-fcb9-469d-abba-aecf3cb7046e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:18.054143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4af30-fcb9-469d-abba-aecf3cb7046e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:18.054641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4af30-fcb9-469d-abba-aecf3cb7046e None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:18.055542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4af30-fcb9-469d-abba-aecf3cb7046e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:18.056048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4af30-fcb9-469d-abba-aecf3cb7046e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:18.057657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4af30-fcb9-469d-abba-aecf3cb7046e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:18.067042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 191/391] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:17 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:18.073071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33677cd-c9f8-47d8-b6da-2310dd0a2170 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:18.131301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:18.145001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2e490-6586-4fae-bbe3-63c7d421d16d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:18.145309 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2e490-6586-4fae-bbe3-63c7d421d16d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:18.145561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2e490-6586-4fae-bbe3-63c7d421d16d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:18.147833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 192/392] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:23:18.159328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215d68e-fdc6-4a51-ae12-66b8d50ae40a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:18.159647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215d68e-fdc6-4a51-ae12-66b8d50ae40a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:18.159877 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215d68e-fdc6-4a51-ae12-66b8d50ae40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:18.192028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6215d68e-fdc6-4a51-ae12-66b8d50ae40a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:18.194868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6215d68e-fdc6-4a51-ae12-66b8d50ae40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:18.254779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33677cd-c9f8-47d8-b6da-2310dd0a2170 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:18.354297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31394d-fcb1-4439-b7d0-952dcff27f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:18.354943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31394d-fcb1-4439-b7d0-952dcff27f47 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:18.355219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31394d-fcb1-4439-b7d0-952dcff27f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:18.355489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31394d-fcb1-4439-b7d0-952dcff27f47 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:18.357289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 193/393] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:23:18.361887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31394d-fcb1-4439-b7d0-952dcff27f47 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:18.363439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31394d-fcb1-4439-b7d0-952dcff27f47 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:18.365509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 201/394] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:18 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:18.371426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215d68e-fdc6-4a51-ae12-66b8d50ae40a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:18.399493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:18.530032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215d68e-fdc6-4a51-ae12-66b8d50ae40a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:18.581035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18589f48-8f7b-40db-86f0-cf2464bd2e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:18.581716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18589f48-8f7b-40db-86f0-cf2464bd2e51 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:18.582061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18589f48-8f7b-40db-86f0-cf2464bd2e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:18.582339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18589f48-8f7b-40db-86f0-cf2464bd2e51 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:18.583466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18589f48-8f7b-40db-86f0-cf2464bd2e51 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:18.585782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18589f48-8f7b-40db-86f0-cf2464bd2e51 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:18.587906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 194/395] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:18 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:18.593594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31394d-fcb1-4439-b7d0-952dcff27f47 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:18.741820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:18.758504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31394d-fcb1-4439-b7d0-952dcff27f47 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:18.811698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18589f48-8f7b-40db-86f0-cf2464bd2e51 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:18.812124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14227c4-e8fe-4cb2-a024-a2885e99bd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:18.812811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14227c4-e8fe-4cb2-a024-a2885e99bd37 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:18.813122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14227c4-e8fe-4cb2-a024-a2885e99bd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:18.813348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14227c4-e8fe-4cb2-a024-a2885e99bd37 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1/groups/0651fc7bb71542c287b073460128eef5/roles/5dc7056ddc154b81939a17474a64f546` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:18.823235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14227c4-e8fe-4cb2-a024-a2885e99bd37 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fc37cae8bab49beb9244a40222ffda1, group_id=0651fc7bb71542c287b073460128eef5, role_id=5dc7056ddc154b81939a17474a64f546)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:18.823235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14227c4-e8fe-4cb2-a024-a2885e99bd37 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:18.839471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 202/396] 10.222.0.100 () {58 vars in 1447 bytes} [Thu Sep 1 17:23:18 2022] PUT /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1/groups/0651fc7bb71542c287b073460128eef5/roles/5dc7056ddc154b81939a17474a64f546 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:18.924756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:18.948729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18589f48-8f7b-40db-86f0-cf2464bd2e51 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:19.011416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb0dcd-7a33-4d66-a3fa-944b2ca323c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:19.012363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb0dcd-7a33-4d66-a3fa-944b2ca323c0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.012785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb0dcd-7a33-4d66-a3fa-944b2ca323c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.013195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb0dcd-7a33-4d66-a3fa-944b2ca323c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.023747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4619ff-a213-4a38-ad9d-f040d1242d67 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.024344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4619ff-a213-4a38-ad9d-f040d1242d67 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.024806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4619ff-a213-4a38-ad9d-f040d1242d67 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.026601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 203/397] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:19.030199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb0dcd-7a33-4d66-a3fa-944b2ca323c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:19.032344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e989d36-f7e9-458b-a1bc-fcb97be3246b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.032861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e989d36-f7e9-458b-a1bc-fcb97be3246b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.033361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e989d36-f7e9-458b-a1bc-fcb97be3246b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cfcb0dcd-7a33-4d66-a3fa-944b2ca323c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:19.036286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:19.037253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:19.038732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 195/398] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:19.045772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb0dcd-7a33-4d66-a3fa-944b2ca323c0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:19.059599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2e989d36-f7e9-458b-a1bc-fcb97be3246b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:19.061322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2e989d36-f7e9-458b-a1bc-fcb97be3246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:19.145330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 204/399] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:19.162040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb0dcd-7a33-4d66-a3fa-944b2ca323c0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:19.199615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481aa087-9373-41fd-9677-8ee96fdd3415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:19.200583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481aa087-9373-41fd-9677-8ee96fdd3415 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.201068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481aa087-9373-41fd-9677-8ee96fdd3415 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.201598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481aa087-9373-41fd-9677-8ee96fdd3415 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.202475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481aa087-9373-41fd-9677-8ee96fdd3415 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:19.202978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481aa087-9373-41fd-9677-8ee96fdd3415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:19.204575 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481aa087-9373-41fd-9677-8ee96fdd3415 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:19.214764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 196/400] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:19 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:19.220712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e989d36-f7e9-458b-a1bc-fcb97be3246b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:19.345029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:19.375320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e989d36-f7e9-458b-a1bc-fcb97be3246b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:19.423653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b11ad9-dea5-4f93-b379-732fe054d40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:19.424519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b11ad9-dea5-4f93-b379-732fe054d40e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.424978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b11ad9-dea5-4f93-b379-732fe054d40e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.425394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b11ad9-dea5-4f93-b379-732fe054d40e None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.430253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b11ad9-dea5-4f93-b379-732fe054d40e None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:19.431888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b11ad9-dea5-4f93-b379-732fe054d40e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:19.433770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 205/401] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:19 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:19.439760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481aa087-9373-41fd-9677-8ee96fdd3415 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:19.449754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:19.477026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f5a96-72bd-41f7-ad9e-8e6b6d4537a6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.477687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f5a96-72bd-41f7-ad9e-8e6b6d4537a6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.478168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f5a96-72bd-41f7-ad9e-8e6b6d4537a6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.480012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 206/402] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:19.495423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05022c-209f-4295-8628-9f2375e37629 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.496035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05022c-209f-4295-8628-9f2375e37629 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.496526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05022c-209f-4295-8628-9f2375e37629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.526772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4c05022c-209f-4295-8628-9f2375e37629 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:19.528006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4c05022c-209f-4295-8628-9f2375e37629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:19.571769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481aa087-9373-41fd-9677-8ee96fdd3415 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:19.605776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 207/403] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:23:19.606617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05022c-209f-4295-8628-9f2375e37629 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:19.610460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d38366-909b-467b-8aa1-50b9e4a1b54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:19.611395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d38366-909b-467b-8aa1-50b9e4a1b54c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.611905 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d38366-909b-467b-8aa1-50b9e4a1b54c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.612400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d38366-909b-467b-8aa1-50b9e4a1b54c None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.613832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d38366-909b-467b-8aa1-50b9e4a1b54c None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:19.615662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d38366-909b-467b-8aa1-50b9e4a1b54c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:19.618018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 197/404] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:19 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:19.626636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d38366-909b-467b-8aa1-50b9e4a1b54c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:19.759319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:19.792820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:19.806809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05022c-209f-4295-8628-9f2375e37629 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:19.850115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d38366-909b-467b-8aa1-50b9e4a1b54c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:19.884436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796d38d-9f2a-4db8-831a-6c36c884e39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:19.885085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796d38d-9f2a-4db8-831a-6c36c884e39d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.885365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796d38d-9f2a-4db8-831a-6c36c884e39d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.885662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796d38d-9f2a-4db8-831a-6c36c884e39d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.899731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796d38d-9f2a-4db8-831a-6c36c884e39d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b796d38d-9f2a-4db8-831a-6c36c884e39d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:19.905704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:19.907441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:19.908845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 208/405] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:23:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:19.912794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796d38d-9f2a-4db8-831a-6c36c884e39d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:19.914817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99aecf9-6b6d-4394-b07d-2cc4925396fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:19.915498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aecf9-6b6d-4394-b07d-2cc4925396fa None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:19.915805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aecf9-6b6d-4394-b07d-2cc4925396fa None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:19.916203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aecf9-6b6d-4394-b07d-2cc4925396fa None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078/groups/0651fc7bb71542c287b073460128eef5/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:19.918335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99aecf9-6b6d-4394-b07d-2cc4925396fa None admin] RBAC: Authorizing `identity:create_grant(project_id=4d0f6555111843d2af0acf5759071078, group_id=0651fc7bb71542c287b073460128eef5, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:19.920387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99aecf9-6b6d-4394-b07d-2cc4925396fa None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:19.951174 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 198/406] 10.222.0.100 () {58 vars in 1447 bytes} [Thu Sep 1 17:23:19 2022] PUT /identity/v3/projects/4d0f6555111843d2af0acf5759071078/groups/0651fc7bb71542c287b073460128eef5/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:19.976034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99aecf9-6b6d-4394-b07d-2cc4925396fa None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:20.037134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796d38d-9f2a-4db8-831a-6c36c884e39d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:20.088072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853aa49-6b5d-462d-b94c-f5084be8344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:20.089161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853aa49-6b5d-462d-b94c-f5084be8344b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.089702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853aa49-6b5d-462d-b94c-f5084be8344b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.090186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853aa49-6b5d-462d-b94c-f5084be8344b None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.091087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853aa49-6b5d-462d-b94c-f5084be8344b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:20.091722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853aa49-6b5d-462d-b94c-f5084be8344b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:20.093741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853aa49-6b5d-462d-b94c-f5084be8344b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:20.106695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 209/407] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:19 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:20.108395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab3615-ba24-4f6d-86a9-dbe369dd5b01 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.108823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab3615-ba24-4f6d-86a9-dbe369dd5b01 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.109381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab3615-ba24-4f6d-86a9-dbe369dd5b01 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.111364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 210/408] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:20.117216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99aecf9-6b6d-4394-b07d-2cc4925396fa None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:20.119684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab3615-ba24-4f6d-86a9-dbe369dd5b01 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:20.123273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:20.165948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5a828-35ef-4c5f-9e33-65362fe04dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:20.166967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5a828-35ef-4c5f-9e33-65362fe04dfc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.166967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5a828-35ef-4c5f-9e33-65362fe04dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.167162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5a828-35ef-4c5f-9e33-65362fe04dfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.175647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:20.186541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5a828-35ef-4c5f-9e33-65362fe04dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-aef5a828-35ef-4c5f-9e33-65362fe04dfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:20.190447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:20.191399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:23:20.192199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:20.198626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 199/409] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:20.199991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698ab99-20bd-48dd-a711-bf72cc7dbd7c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.200283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698ab99-20bd-48dd-a711-bf72cc7dbd7c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.200558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698ab99-20bd-48dd-a711-bf72cc7dbd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.220771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a698ab99-20bd-48dd-a711-bf72cc7dbd7c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:20.221986 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a698ab99-20bd-48dd-a711-bf72cc7dbd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:20.278916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab3615-ba24-4f6d-86a9-dbe369dd5b01 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:20.309687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 200/410] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:20.309687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698ab99-20bd-48dd-a711-bf72cc7dbd7c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:20.349353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4b9c6-4d67-45a1-915c-8e3b2f5f0da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:20.354619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4b9c6-4d67-45a1-915c-8e3b2f5f0da7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.354619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4b9c6-4d67-45a1-915c-8e3b2f5f0da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.354619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4b9c6-4d67-45a1-915c-8e3b2f5f0da7 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.358436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4b9c6-4d67-45a1-915c-8e3b2f5f0da7 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:20.360247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4b9c6-4d67-45a1-915c-8e3b2f5f0da7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:20.362712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 211/411] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:20 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:20.368460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4b9c6-4d67-45a1-915c-8e3b2f5f0da7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:20.449204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698ab99-20bd-48dd-a711-bf72cc7dbd7c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:20.475722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:20.488671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b79df-03c5-41b4-a7c7-b2004435e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:20.489287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b79df-03c5-41b4-a7c7-b2004435e911 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.489583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b79df-03c5-41b4-a7c7-b2004435e911 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.489870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b79df-03c5-41b4-a7c7-b2004435e911 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.493614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b79df-03c5-41b4-a7c7-b2004435e911 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:20.493614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b79df-03c5-41b4-a7c7-b2004435e911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:20.493614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b79df-03c5-41b4-a7c7-b2004435e911 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:20.501797 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 201/412] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:20 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:20.508325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b79df-03c5-41b4-a7c7-b2004435e911 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:20.512879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:20.568558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4b9c6-4d67-45a1-915c-8e3b2f5f0da7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:20.632301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f609ec9-99e2-4637-a1d4-fe819ea189b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:20.632945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f609ec9-99e2-4637-a1d4-fe819ea189b6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.633222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f609ec9-99e2-4637-a1d4-fe819ea189b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.633520 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f609ec9-99e2-4637-a1d4-fe819ea189b6 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.636753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f609ec9-99e2-4637-a1d4-fe819ea189b6 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:20.638251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f609ec9-99e2-4637-a1d4-fe819ea189b6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:20.640076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 212/413] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:20 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:20.643635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f609ec9-99e2-4637-a1d4-fe819ea189b6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:20.679719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b79df-03c5-41b4-a7c7-b2004435e911 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:20.720985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2d269-2518-4117-aefb-dd6d52ba07f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:20.721898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2d269-2518-4117-aefb-dd6d52ba07f7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.722186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2d269-2518-4117-aefb-dd6d52ba07f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.722460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2d269-2518-4117-aefb-dd6d52ba07f7 None admin] PATH_INFO: `/v3/groups/bf6fe40707c04f3abf1ca19d39e10e2c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.723581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2d269-2518-4117-aefb-dd6d52ba07f7 None admin] RBAC: Authorizing `identity:get_group(group_id=bf6fe40707c04f3abf1ca19d39e10e2c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:20.726154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2d269-2518-4117-aefb-dd6d52ba07f7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:20.729354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 202/414] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:20 2022] GET /identity/v3/groups/bf6fe40707c04f3abf1ca19d39e10e2c => generated 230 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:20.730449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2d269-2518-4117-aefb-dd6d52ba07f7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:20.784439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f609ec9-99e2-4637-a1d4-fe819ea189b6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:20.805864 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:20.826573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b8531-0ba0-4519-8585-d3383cf1cdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:20.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b8531-0ba0-4519-8585-d3383cf1cdad None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.828109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b8531-0ba0-4519-8585-d3383cf1cdad None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.828591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b8531-0ba0-4519-8585-d3383cf1cdad None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.837594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:20.843883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b8531-0ba0-4519-8585-d3383cf1cdad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-190b8531-0ba0-4519-8585-d3383cf1cdad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:20.849495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:20.850505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:20.851317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:20.854305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 213/415] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:20.855302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b8531-0ba0-4519-8585-d3383cf1cdad None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:20.856650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2d269-2518-4117-aefb-dd6d52ba07f7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:20.902171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f14aa-c8e3-4bae-9358-b07536e88428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:20.903216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f14aa-c8e3-4bae-9358-b07536e88428 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:20.903701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f14aa-c8e3-4bae-9358-b07536e88428 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:20.904173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f14aa-c8e3-4bae-9358-b07536e88428 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078/groups/0651fc7bb71542c287b073460128eef5/roles/5dc7056ddc154b81939a17474a64f546` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:20.906903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f14aa-c8e3-4bae-9358-b07536e88428 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d0f6555111843d2af0acf5759071078, group_id=0651fc7bb71542c287b073460128eef5, role_id=5dc7056ddc154b81939a17474a64f546)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:20.908761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f14aa-c8e3-4bae-9358-b07536e88428 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:20.937023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 203/416] 10.222.0.100 () {58 vars in 1447 bytes} [Thu Sep 1 17:23:20 2022] PUT /identity/v3/projects/4d0f6555111843d2af0acf5759071078/groups/0651fc7bb71542c287b073460128eef5/roles/5dc7056ddc154b81939a17474a64f546 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:20.937023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f14aa-c8e3-4bae-9358-b07536e88428 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:21.027380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b8531-0ba0-4519-8585-d3383cf1cdad None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:21.062736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f14aa-c8e3-4bae-9358-b07536e88428 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:21.080576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:21.111509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77501e-a215-4281-8bbf-9b16cf4d6307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:21.113591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77501e-a215-4281-8bbf-9b16cf4d6307 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:21.114420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77501e-a215-4281-8bbf-9b16cf4d6307 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:21.115766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77501e-a215-4281-8bbf-9b16cf4d6307 None admin] PATH_INFO: `/v3/projects/392e00333a6b41639a27c8ea706c3db5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:21.118639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77501e-a215-4281-8bbf-9b16cf4d6307 None admin] RBAC: Authorizing `identity:get_project(project_id=392e00333a6b41639a27c8ea706c3db5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:21.122113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77501e-a215-4281-8bbf-9b16cf4d6307 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:21.125916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 214/417] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:20 2022] GET /identity/v3/projects/392e00333a6b41639a27c8ea706c3db5 => generated 345 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:21.129663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73480e-d4ad-42ce-b5b5-99b6da5f1589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:21.131288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77501e-a215-4281-8bbf-9b16cf4d6307 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:21.134248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73480e-d4ad-42ce-b5b5-99b6da5f1589 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:21.135067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73480e-d4ad-42ce-b5b5-99b6da5f1589 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:21.136308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73480e-d4ad-42ce-b5b5-99b6da5f1589 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:21.137221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73480e-d4ad-42ce-b5b5-99b6da5f1589 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:21.138078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73480e-d4ad-42ce-b5b5-99b6da5f1589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:21.140853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73480e-d4ad-42ce-b5b5-99b6da5f1589 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:21.159846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 204/418] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:20 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:21.161738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4ad3b-6338-450c-a3ae-080ccb86b792 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:21.162317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4ad3b-6338-450c-a3ae-080ccb86b792 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:21.163810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4ad3b-6338-450c-a3ae-080ccb86b792 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:21.165526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 205/419] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:21 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:21.168753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4ad3b-6338-450c-a3ae-080ccb86b792 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:21.238372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:21.279731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77501e-a215-4281-8bbf-9b16cf4d6307 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:21.323318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841111a-577d-41e4-ac71-ab813b0c7cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:21.323940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841111a-577d-41e4-ac71-ab813b0c7cac None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:21.324190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841111a-577d-41e4-ac71-ab813b0c7cac None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:21.324530 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841111a-577d-41e4-ac71-ab813b0c7cac None admin] PATH_INFO: `/v3/projects/392e00333a6b41639a27c8ea706c3db5/groups/bf6fe40707c04f3abf1ca19d39e10e2c/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:21.327040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841111a-577d-41e4-ac71-ab813b0c7cac None admin] RBAC: Authorizing `identity:create_grant(project_id=392e00333a6b41639a27c8ea706c3db5, group_id=bf6fe40707c04f3abf1ca19d39e10e2c, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:21.328520 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841111a-577d-41e4-ac71-ab813b0c7cac None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:21.329687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4ad3b-6338-450c-a3ae-080ccb86b792 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:21.358002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 215/420] 10.222.0.100 () {58 vars in 1447 bytes} [Thu Sep 1 17:23:21 2022] PUT /identity/v3/projects/392e00333a6b41639a27c8ea706c3db5/groups/bf6fe40707c04f3abf1ca19d39e10e2c/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:21.359518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790d7d-3fdf-45b1-844d-a73ca1c26145 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:21.359928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790d7d-3fdf-45b1-844d-a73ca1c26145 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:21.360251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790d7d-3fdf-45b1-844d-a73ca1c26145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:21.375084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3acbc-8883-496b-88a2-dd088f58d2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:21.376085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3acbc-8883-496b-88a2-dd088f58d2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:21.376564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3acbc-8883-496b-88a2-dd088f58d2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:21.377035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3acbc-8883-496b-88a2-dd088f58d2b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:21.390167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-40790d7d-3fdf-45b1-844d-a73ca1c26145 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:21.391790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-40790d7d-3fdf-45b1-844d-a73ca1c26145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:21.442013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:21.471372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3acbc-8883-496b-88a2-dd088f58d2b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:21.473241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3acbc-8883-496b-88a2-dd088f58d2b5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:21.536674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 216/421] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:21.601128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 206/422] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:21 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:23:21.608614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40790d7d-3fdf-45b1-844d-a73ca1c26145 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:21.683066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:21.739589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40790d7d-3fdf-45b1-844d-a73ca1c26145 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:21.776947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244927b6-0b44-4b5b-8574-7587c400fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:21.777616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244927b6-0b44-4b5b-8574-7587c400fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:21.777901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244927b6-0b44-4b5b-8574-7587c400fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:21.778175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244927b6-0b44-4b5b-8574-7587c400fdd1 None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:21.779382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244927b6-0b44-4b5b-8574-7587c400fdd1 None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:21.780838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244927b6-0b44-4b5b-8574-7587c400fdd1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:21.782859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 217/423] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:21 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:21.789122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3acbc-8883-496b-88a2-dd088f58d2b5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:21.907594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:21.940350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3acbc-8883-496b-88a2-dd088f58d2b5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:21.942395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244927b6-0b44-4b5b-8574-7587c400fdd1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:21.984038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa86d1-9b4c-4121-82b2-542b2e73c703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:21.984680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa86d1-9b4c-4121-82b2-542b2e73c703 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:21.984958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa86d1-9b4c-4121-82b2-542b2e73c703 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:21.985230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa86d1-9b4c-4121-82b2-542b2e73c703 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:21.986368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa86d1-9b4c-4121-82b2-542b2e73c703 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:21.987820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa86d1-9b4c-4121-82b2-542b2e73c703 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:21.989753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 207/424] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:21 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:22.002591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa86d1-9b4c-4121-82b2-542b2e73c703 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:22.119183 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244927b6-0b44-4b5b-8574-7587c400fdd1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:22.161667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:22.184343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62169789-4951-49de-ab5f-b1eaf345311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:22.184985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62169789-4951-49de-ab5f-b1eaf345311a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:22.185258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62169789-4951-49de-ab5f-b1eaf345311a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:22.185527 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62169789-4951-49de-ab5f-b1eaf345311a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:22.190869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa86d1-9b4c-4121-82b2-542b2e73c703 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:22.211836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62169789-4951-49de-ab5f-b1eaf345311a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-62169789-4951-49de-ab5f-b1eaf345311a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:22.216767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:22.217659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:22.218620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 218/425] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:23:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:22.224755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62169789-4951-49de-ab5f-b1eaf345311a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:22.261030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb59284-fbb9-46d7-aa68-b813e17214dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:22.261666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb59284-fbb9-46d7-aa68-b813e17214dd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:22.261942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb59284-fbb9-46d7-aa68-b813e17214dd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:22.262206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb59284-fbb9-46d7-aa68-b813e17214dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:22.262928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb59284-fbb9-46d7-aa68-b813e17214dd None admin] RBAC: Adding query filter params (group.id=0651fc7bb71542c287b073460128eef5, role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=6fc37cae8bab49beb9244a40222ffda1) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:22.263230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb59284-fbb9-46d7-aa68-b813e17214dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:22.264717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb59284-fbb9-46d7-aa68-b813e17214dd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:22.271520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 208/426] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:22 2022] GET /identity/v3/role_assignments?group.id=0651fc7bb71542c287b073460128eef5&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=90949fa670c2407ca92b16156e52f47f => generated 613 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:22.273869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b333ebd-c2cc-4488-aefb-123ff5f77733 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:22.274340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b333ebd-c2cc-4488-aefb-123ff5f77733 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:22.274622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b333ebd-c2cc-4488-aefb-123ff5f77733 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:22.280868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 209/427] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:22.282502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd176f5b-c88f-43f1-a4f0-a1b0c6f836eb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:22.282791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd176f5b-c88f-43f1-a4f0-a1b0c6f836eb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:22.283064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd176f5b-c88f-43f1-a4f0-a1b0c6f836eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:22.313299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fd176f5b-c88f-43f1-a4f0-a1b0c6f836eb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:22.314640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fd176f5b-c88f-43f1-a4f0-a1b0c6f836eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:22.321966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:22.433561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62169789-4951-49de-ab5f-b1eaf345311a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:22.448328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 210/428] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:23:22.471604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcbe9e-4f02-4c52-8791-6f47a00f4efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:22.472207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcbe9e-4f02-4c52-8791-6f47a00f4efa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:22.472484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcbe9e-4f02-4c52-8791-6f47a00f4efa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:22.472753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcbe9e-4f02-4c52-8791-6f47a00f4efa None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:22.473410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcbe9e-4f02-4c52-8791-6f47a00f4efa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:22.473739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcbe9e-4f02-4c52-8791-6f47a00f4efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:22.475146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcbe9e-4f02-4c52-8791-6f47a00f4efa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:22.484073 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 219/429] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:22 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:22.489731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd176f5b-c88f-43f1-a4f0-a1b0c6f836eb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:22.596341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd176f5b-c88f-43f1-a4f0-a1b0c6f836eb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:22.632395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca880b-0358-4b05-b910-d5ca670f61e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:22.633007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca880b-0358-4b05-b910-d5ca670f61e9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:22.633284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca880b-0358-4b05-b910-d5ca670f61e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:22.633552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca880b-0358-4b05-b910-d5ca670f61e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:22.642489 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:22.721748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca880b-0358-4b05-b910-d5ca670f61e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:22.723111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca880b-0358-4b05-b910-d5ca670f61e9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:22.775953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcbe9e-4f02-4c52-8791-6f47a00f4efa None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:22.853010 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:22.862727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 211/430] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:22 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:22.868970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca880b-0358-4b05-b910-d5ca670f61e9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:22.925411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:22.936624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcbe9e-4f02-4c52-8791-6f47a00f4efa None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:22.988157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dca682-a61a-4eda-bdef-1891fe8d3974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:22.989346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dca682-a61a-4eda-bdef-1891fe8d3974 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:22.989831 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dca682-a61a-4eda-bdef-1891fe8d3974 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:22.990278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dca682-a61a-4eda-bdef-1891fe8d3974 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.013977 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dca682-a61a-4eda-bdef-1891fe8d3974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:23.023009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca880b-0358-4b05-b910-d5ca670f61e9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f1dca682-a61a-4eda-bdef-1891fe8d3974 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:23.024850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:23.025790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:23:23.026854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:23.029599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 220/431] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:23.036136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dca682-a61a-4eda-bdef-1891fe8d3974 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:23.069302 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab81a74-b2f6-4a37-8953-a6424743e06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:23.071171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab81a74-b2f6-4a37-8953-a6424743e06a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:23.071464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab81a74-b2f6-4a37-8953-a6424743e06a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:23.071740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab81a74-b2f6-4a37-8953-a6424743e06a None admin] PATH_INFO: `/v3/projects/6fc37cae8bab49beb9244a40222ffda1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.073007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab81a74-b2f6-4a37-8953-a6424743e06a None admin] RBAC: Authorizing `identity:get_project(project_id=6fc37cae8bab49beb9244a40222ffda1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:23.074530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab81a74-b2f6-4a37-8953-a6424743e06a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:23.079764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 212/432] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:22 2022] GET /identity/v3/projects/6fc37cae8bab49beb9244a40222ffda1 => generated 299 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:23.082321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab81a74-b2f6-4a37-8953-a6424743e06a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:23.176977 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dca682-a61a-4eda-bdef-1891fe8d3974 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:23.188122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:23.201634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab81a74-b2f6-4a37-8953-a6424743e06a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:23.211491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:23.228405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521bfbe-0542-4a04-bf47-483a13b2861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:23.228995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521bfbe-0542-4a04-bf47-483a13b2861e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:23.229352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521bfbe-0542-4a04-bf47-483a13b2861e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:23.229531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521bfbe-0542-4a04-bf47-483a13b2861e None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.230245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521bfbe-0542-4a04-bf47-483a13b2861e None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:23.230547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521bfbe-0542-4a04-bf47-483a13b2861e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:23.232057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521bfbe-0542-4a04-bf47-483a13b2861e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:23.243578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 221/433] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:23:23 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:23.248467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f60f5a-e576-42b1-970a-042302dc6018 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:23.248762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f60f5a-e576-42b1-970a-042302dc6018 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:23.249044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f60f5a-e576-42b1-970a-042302dc6018 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.249363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d378d74-a729-46f8-bb70-6102d91b6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:23.250271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d378d74-a729-46f8-bb70-6102d91b6e32 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:23.250746 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d378d74-a729-46f8-bb70-6102d91b6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:23.251211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d378d74-a729-46f8-bb70-6102d91b6e32 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.253489 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d378d74-a729-46f8-bb70-6102d91b6e32 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:23.254727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 222/434] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:23:23.255372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d378d74-a729-46f8-bb70-6102d91b6e32 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:23.255621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f60f5a-e576-42b1-970a-042302dc6018 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:23.257785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 213/435] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:23 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:23.259228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63f79d-bb78-4a4a-9f00-607732afb5c3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:23.259693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63f79d-bb78-4a4a-9f00-607732afb5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:23.259968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63f79d-bb78-4a4a-9f00-607732afb5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.291488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2a63f79d-bb78-4a4a-9f00-607732afb5c3 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:23.292710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2a63f79d-bb78-4a4a-9f00-607732afb5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:23.371826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 214/436] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:23.374484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63f79d-bb78-4a4a-9f00-607732afb5c3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:23.387962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f60f5a-e576-42b1-970a-042302dc6018 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:23.429369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec368492-f0ec-4983-b20c-66dad0006264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:23.430498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec368492-f0ec-4983-b20c-66dad0006264 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:23.431015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec368492-f0ec-4983-b20c-66dad0006264 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:23.431486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec368492-f0ec-4983-b20c-66dad0006264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.472902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:23.514207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec368492-f0ec-4983-b20c-66dad0006264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:23.515172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63f79d-bb78-4a4a-9f00-607732afb5c3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:23.516208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec368492-f0ec-4983-b20c-66dad0006264 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:23.517668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:23.562078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bda84-65ca-47f7-b767-b6d4a9186626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:23.562697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bda84-65ca-47f7-b767-b6d4a9186626 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:23.562978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bda84-65ca-47f7-b767-b6d4a9186626 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:23.563250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bda84-65ca-47f7-b767-b6d4a9186626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.564017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bda84-65ca-47f7-b767-b6d4a9186626 None admin] RBAC: Adding query filter params (group.id=0651fc7bb71542c287b073460128eef5, role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=6fc37cae8bab49beb9244a40222ffda1) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:23.564891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bda84-65ca-47f7-b767-b6d4a9186626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:23.566362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bda84-65ca-47f7-b767-b6d4a9186626 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:23.577637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 215/437] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:23 2022] GET /identity/v3/role_assignments?group.id=0651fc7bb71542c287b073460128eef5&scope.project.id=6fc37cae8bab49beb9244a40222ffda1&role.id=5dc7056ddc154b81939a17474a64f546 => generated 613 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:23.645902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 223/438] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:23.649929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bda84-65ca-47f7-b767-b6d4a9186626 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:23.653567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec368492-f0ec-4983-b20c-66dad0006264 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:23.730594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:23.777798 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bda84-65ca-47f7-b767-b6d4a9186626 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:23.778043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec368492-f0ec-4983-b20c-66dad0006264 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:23.820377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce5d51-1214-4aa3-ba3b-14152fa12807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:23.820757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a4f3-5988-4002-bb59-6ccd7dded217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:23.821028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce5d51-1214-4aa3-ba3b-14152fa12807 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:23.821310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce5d51-1214-4aa3-ba3b-14152fa12807 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:23.821406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a4f3-5988-4002-bb59-6ccd7dded217 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:23.821644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce5d51-1214-4aa3-ba3b-14152fa12807 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.821755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a4f3-5988-4002-bb59-6ccd7dded217 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:23.821995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a4f3-5988-4002-bb59-6ccd7dded217 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:23.822870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce5d51-1214-4aa3-ba3b-14152fa12807 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:23.824317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce5d51-1214-4aa3-ba3b-14152fa12807 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:23.826678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 224/439] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:23 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:23.831738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce5d51-1214-4aa3-ba3b-14152fa12807 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:23.838377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4a4f3-5988-4002-bb59-6ccd7dded217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-afe4a4f3-5988-4002-bb59-6ccd7dded217 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:23.844033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:23.844962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:23.845811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:23.846726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 216/440] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:23:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:23.853719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a4f3-5988-4002-bb59-6ccd7dded217 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:23.890500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:23.961260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce5d51-1214-4aa3-ba3b-14152fa12807 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:23.980541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a4f3-5988-4002-bb59-6ccd7dded217 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:24.010415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657904ea-0c30-4011-8837-41d1808f0339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.011056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657904ea-0c30-4011-8837-41d1808f0339 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.011341 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657904ea-0c30-4011-8837-41d1808f0339 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.011614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657904ea-0c30-4011-8837-41d1808f0339 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.012813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657904ea-0c30-4011-8837-41d1808f0339 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:24.014323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657904ea-0c30-4011-8837-41d1808f0339 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:24.016187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 225/441] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:23 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:24.019366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b49c95-a7b1-446c-8264-ba894d942a18 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.019893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b49c95-a7b1-446c-8264-ba894d942a18 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.020338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b49c95-a7b1-446c-8264-ba894d942a18 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.022784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 226/442] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:24 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:24.023946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b49c95-a7b1-446c-8264-ba894d942a18 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:24.029841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949733d-88b2-4f84-8f26-d5cf08328cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.030412 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949733d-88b2-4f84-8f26-d5cf08328cec None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.030689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949733d-88b2-4f84-8f26-d5cf08328cec None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.030963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949733d-88b2-4f84-8f26-d5cf08328cec None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.031526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:24.031670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949733d-88b2-4f84-8f26-d5cf08328cec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:24.031931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949733d-88b2-4f84-8f26-d5cf08328cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:24.033467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949733d-88b2-4f84-8f26-d5cf08328cec None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:24.042695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 217/443] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:23 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:24.043768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26fba-5824-43cc-a63f-83068785da1d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.044059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26fba-5824-43cc-a63f-83068785da1d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.044344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26fba-5824-43cc-a63f-83068785da1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.063348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b8c26fba-5824-43cc-a63f-83068785da1d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:24.064556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b8c26fba-5824-43cc-a63f-83068785da1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:24.140873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 218/444] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:24.140873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c26fba-5824-43cc-a63f-83068785da1d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:24.155308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b49c95-a7b1-446c-8264-ba894d942a18 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:24.166750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:24.194955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e826be-afc3-4378-9751-d2d623ca862f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.195551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e826be-afc3-4378-9751-d2d623ca862f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.195831 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e826be-afc3-4378-9751-d2d623ca862f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.196096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e826be-afc3-4378-9751-d2d623ca862f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.196848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e826be-afc3-4378-9751-d2d623ca862f None admin] RBAC: Adding query filter params (group.id=0651fc7bb71542c287b073460128eef5, role.id=90949fa670c2407ca92b16156e52f47f, scope.project.id=4d0f6555111843d2af0acf5759071078) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:24.197157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e826be-afc3-4378-9751-d2d623ca862f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:24.198623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e826be-afc3-4378-9751-d2d623ca862f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:24.213613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 227/445] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:24 2022] GET /identity/v3/role_assignments?group.id=0651fc7bb71542c287b073460128eef5&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=90949fa670c2407ca92b16156e52f47f => generated 613 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:24.280898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c26fba-5824-43cc-a63f-83068785da1d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:24.322525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b51580-8340-4fa3-81e1-245a98d0bfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.323141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b51580-8340-4fa3-81e1-245a98d0bfef None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.323416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b51580-8340-4fa3-81e1-245a98d0bfef None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.323681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b51580-8340-4fa3-81e1-245a98d0bfef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.400026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b51580-8340-4fa3-81e1-245a98d0bfef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:24.401775 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b51580-8340-4fa3-81e1-245a98d0bfef None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:24.405660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e826be-afc3-4378-9751-d2d623ca862f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:24.425974 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:24.484389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:24.526792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e826be-afc3-4378-9751-d2d623ca862f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:24.527017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 219/446] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:24 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:24.532774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b51580-8340-4fa3-81e1-245a98d0bfef None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:24.565893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d32a6-b1a0-41e2-aa7a-32201b52be64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.566691 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d32a6-b1a0-41e2-aa7a-32201b52be64 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.566905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d32a6-b1a0-41e2-aa7a-32201b52be64 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.567176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d32a6-b1a0-41e2-aa7a-32201b52be64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.581885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d32a6-b1a0-41e2-aa7a-32201b52be64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e17d32a6-b1a0-41e2-aa7a-32201b52be64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:24.587534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:24.588476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:24.589673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 228/447] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:24.594538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d32a6-b1a0-41e2-aa7a-32201b52be64 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:24.646679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b51580-8340-4fa3-81e1-245a98d0bfef None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:24.669885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:24.684515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc93d4-568e-4e2e-97fb-b4adfdfa7e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.685130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc93d4-568e-4e2e-97fb-b4adfdfa7e17 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.685409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc93d4-568e-4e2e-97fb-b4adfdfa7e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.685713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc93d4-568e-4e2e-97fb-b4adfdfa7e17 None admin] PATH_INFO: `/v3/projects/4d0f6555111843d2af0acf5759071078` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.686805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc93d4-568e-4e2e-97fb-b4adfdfa7e17 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0f6555111843d2af0acf5759071078)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:24.688233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc93d4-568e-4e2e-97fb-b4adfdfa7e17 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:24.689874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 220/448] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:24 2022] GET /identity/v3/projects/4d0f6555111843d2af0acf5759071078 => generated 303 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:24.695339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc93d4-568e-4e2e-97fb-b4adfdfa7e17 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:24.702257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d32a6-b1a0-41e2-aa7a-32201b52be64 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:24.737062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:24.741995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312022f4-e611-4b4d-9a0c-0e302495ccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.742623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312022f4-e611-4b4d-9a0c-0e302495ccd8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.743049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312022f4-e611-4b4d-9a0c-0e302495ccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.743320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312022f4-e611-4b4d-9a0c-0e302495ccd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.743997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312022f4-e611-4b4d-9a0c-0e302495ccd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:24.744300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312022f4-e611-4b4d-9a0c-0e302495ccd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:24.745714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312022f4-e611-4b4d-9a0c-0e302495ccd8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:24.754380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 229/449] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:24 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:24.760293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312022f4-e611-4b4d-9a0c-0e302495ccd8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:24.805189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc93d4-568e-4e2e-97fb-b4adfdfa7e17 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:24.841146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfef5c-8881-4edb-ae87-082038f52a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.841798 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfef5c-8881-4edb-ae87-082038f52a38 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.842078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfef5c-8881-4edb-ae87-082038f52a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.842353 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfef5c-8881-4edb-ae87-082038f52a38 None admin] PATH_INFO: `/v3/groups/0651fc7bb71542c287b073460128eef5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.843501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfef5c-8881-4edb-ae87-082038f52a38 None admin] RBAC: Authorizing `identity:get_group(group_id=0651fc7bb71542c287b073460128eef5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:24.844954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfef5c-8881-4edb-ae87-082038f52a38 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:24.846555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 221/450] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:24 2022] GET /identity/v3/groups/0651fc7bb71542c287b073460128eef5 => generated 227 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:24.852927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfef5c-8881-4edb-ae87-082038f52a38 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:24.867283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312022f4-e611-4b4d-9a0c-0e302495ccd8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:24.901383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba1fc9-f8a5-4051-bbe8-97d8a339a8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.902019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba1fc9-f8a5-4051-bbe8-97d8a339a8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.902308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba1fc9-f8a5-4051-bbe8-97d8a339a8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.902584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba1fc9-f8a5-4051-bbe8-97d8a339a8e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.909242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:24.952935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfef5c-8881-4edb-ae87-082038f52a38 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:24.962006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:24.972692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba1fc9-f8a5-4051-bbe8-97d8a339a8e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:24.974178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba1fc9-f8a5-4051-bbe8-97d8a339a8e2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:24.988276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e411b-9544-41f8-9436-351b796df158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:24.988871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e411b-9544-41f8-9436-351b796df158 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:24.989157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e411b-9544-41f8-9436-351b796df158 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:24.989425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e411b-9544-41f8-9436-351b796df158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:24.990207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e411b-9544-41f8-9436-351b796df158 None admin] RBAC: Adding query filter params (group.id=0651fc7bb71542c287b073460128eef5, role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=4d0f6555111843d2af0acf5759071078) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:24.990638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e411b-9544-41f8-9436-351b796df158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:24.991969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e411b-9544-41f8-9436-351b796df158 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:24.997764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 222/451] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:24 2022] GET /identity/v3/role_assignments?group.id=0651fc7bb71542c287b073460128eef5&scope.project.id=4d0f6555111843d2af0acf5759071078&role.id=5dc7056ddc154b81939a17474a64f546 => generated 613 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:25.079418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 230/452] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:24 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:23:25.087643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e411b-9544-41f8-9436-351b796df158 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:25.195946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e411b-9544-41f8-9436-351b796df158 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:25.217782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:25.232895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9031d-b558-412e-a180-cb00b2a52584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:25.233537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9031d-b558-412e-a180-cb00b2a52584 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:25.233844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9031d-b558-412e-a180-cb00b2a52584 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:25.234117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9031d-b558-412e-a180-cb00b2a52584 None admin] PATH_INFO: `/v3/projects/392e00333a6b41639a27c8ea706c3db5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:25.235268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9031d-b558-412e-a180-cb00b2a52584 None admin] RBAC: Authorizing `identity:get_project(project_id=392e00333a6b41639a27c8ea706c3db5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:25.236715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9031d-b558-412e-a180-cb00b2a52584 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:25.238392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 223/453] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:23:25 2022] GET /identity/v3/projects/392e00333a6b41639a27c8ea706c3db5 => generated 345 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:25.244991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba1fc9-f8a5-4051-bbe8-97d8a339a8e2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:25.344892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:25.356100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba1fc9-f8a5-4051-bbe8-97d8a339a8e2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:25.391319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9d75c-074c-45c9-be68-3ca3446d61c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:25.391943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9d75c-074c-45c9-be68-3ca3446d61c8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:25.392224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9d75c-074c-45c9-be68-3ca3446d61c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:25.392493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9d75c-074c-45c9-be68-3ca3446d61c8 None admin] PATH_INFO: `/v3/groups/bf6fe40707c04f3abf1ca19d39e10e2c` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:25.393620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9d75c-074c-45c9-be68-3ca3446d61c8 None admin] RBAC: Authorizing `identity:get_group(group_id=bf6fe40707c04f3abf1ca19d39e10e2c)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:25.395146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9d75c-074c-45c9-be68-3ca3446d61c8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:25.396631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 231/454] 10.222.0.100 () {56 vars in 1185 bytes} [Thu Sep 1 17:23:25 2022] GET /identity/v3/groups/bf6fe40707c04f3abf1ca19d39e10e2c => generated 230 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:25.404014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9031d-b558-412e-a180-cb00b2a52584 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:25.507821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9031d-b558-412e-a180-cb00b2a52584 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:25.544949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63f0fa-d89c-4ded-af27-785ae36458f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:25.545542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63f0fa-d89c-4ded-af27-785ae36458f1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:25.545853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63f0fa-d89c-4ded-af27-785ae36458f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:25.546125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63f0fa-d89c-4ded-af27-785ae36458f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:25.546838 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63f0fa-d89c-4ded-af27-785ae36458f1 None admin] RBAC: Adding query filter params (group.id=bf6fe40707c04f3abf1ca19d39e10e2c, role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=392e00333a6b41639a27c8ea706c3db5) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:25.547140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63f0fa-d89c-4ded-af27-785ae36458f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:25.548556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63f0fa-d89c-4ded-af27-785ae36458f1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:25.555243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 224/455] 10.222.0.100 () {56 vars in 1381 bytes} [Thu Sep 1 17:23:25 2022] GET /identity/v3/role_assignments?group.id=bf6fe40707c04f3abf1ca19d39e10e2c&scope.project.id=392e00333a6b41639a27c8ea706c3db5&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 613 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:28.260360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecfddc-3389-4752-8b22-d6d8e32e6bd7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.261168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecfddc-3389-4752-8b22-d6d8e32e6bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.261168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecfddc-3389-4752-8b22-d6d8e32e6bd7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.262624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 232/456] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:28.269070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f561c0-bae9-4a75-b680-db886e4c1eb9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.270536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f561c0-bae9-4a75-b680-db886e4c1eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.271387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f561c0-bae9-4a75-b680-db886e4c1eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.315414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-65f561c0-bae9-4a75-b680-db886e4c1eb9 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:28.317526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-65f561c0-bae9-4a75-b680-db886e4c1eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:28.379687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:28.393719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c786aaf-092c-428c-9e78-d3524bc992e2 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.394373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c786aaf-092c-428c-9e78-d3524bc992e2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.394880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c786aaf-092c-428c-9e78-d3524bc992e2 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.398235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 233/457] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:28 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:28.407216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 225/458] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:28.410505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855a25-9370-4158-9b33-b2afe1f59d88 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.410969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855a25-9370-4158-9b33-b2afe1f59d88 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.411387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855a25-9370-4158-9b33-b2afe1f59d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.440360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7b855a25-9370-4158-9b33-b2afe1f59d88 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:28.441873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7b855a25-9370-4158-9b33-b2afe1f59d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:28.545756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 234/459] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:28.673196 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc50663-8a27-4e95-9301-9540ec0b0b4c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.673744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc50663-8a27-4e95-9301-9540ec0b0b4c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.674193 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc50663-8a27-4e95-9301-9540ec0b0b4c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.675683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 226/460] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:28 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:28.680846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb257dee-9412-4126-962b-b4366c9b2a54 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.681321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb257dee-9412-4126-962b-b4366c9b2a54 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.681762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb257dee-9412-4126-962b-b4366c9b2a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.710886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fb257dee-9412-4126-962b-b4366c9b2a54 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:28.712356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fb257dee-9412-4126-962b-b4366c9b2a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:28.735394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc50663-8a27-4e95-9301-9540ec0b0b4c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:28.790444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 235/461] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:28.839810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3d41f-ae25-4577-b5c4-ae5a97f8a055 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.840310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3d41f-ae25-4577-b5c4-ae5a97f8a055 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.840752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3d41f-ae25-4577-b5c4-ae5a97f8a055 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.842262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 236/462] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:23:28.847257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70563b-c28f-4a72-a525-b30fba631e58 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.847732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70563b-c28f-4a72-a525-b30fba631e58 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.848152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70563b-c28f-4a72-a525-b30fba631e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.875449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9d70563b-c28f-4a72-a525-b30fba631e58 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:28.877458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9d70563b-c28f-4a72-a525-b30fba631e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:28.893360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:28.912127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc50663-8a27-4e95-9301-9540ec0b0b4c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:28.956713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 237/463] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:23:28.957757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70563b-c28f-4a72-a525-b30fba631e58 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:28.958012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd1163-adaa-4d73-a199-566ed7ed5ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:28.958802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd1163-adaa-4d73-a199-566ed7ed5ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:28.959227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd1163-adaa-4d73-a199-566ed7ed5ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:28.960374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd1163-adaa-4d73-a199-566ed7ed5ef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:28.988318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd1163-adaa-4d73-a199-566ed7ed5ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:28.989813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd1163-adaa-4d73-a199-566ed7ed5ef8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-57cd1163-adaa-4d73-a199-566ed7ed5ef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:29.020033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:29.021078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:29.023453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:29.023453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:29.023453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:29.023453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 227/464] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:29.030760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd1163-adaa-4d73-a199-566ed7ed5ef8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:29.102582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70563b-c28f-4a72-a525-b30fba631e58 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:29.120968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:29.143937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ce7c5-c66d-4298-93af-9c3289ca600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:29.144898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ce7c5-c66d-4298-93af-9c3289ca600e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:29.145359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ce7c5-c66d-4298-93af-9c3289ca600e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:29.145885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ce7c5-c66d-4298-93af-9c3289ca600e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:29.157518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ce7c5-c66d-4298-93af-9c3289ca600e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:29.159288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ce7c5-c66d-4298-93af-9c3289ca600e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:29.169333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-087ce7c5-c66d-4298-93af-9c3289ca600e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:29.177132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:29.178761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:29.179729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:29.179729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:29.179729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:29.181347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 238/465] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:29.182353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ce7c5-c66d-4298-93af-9c3289ca600e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:29.201411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd1163-adaa-4d73-a199-566ed7ed5ef8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:29.239992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b642d-7554-4cee-b055-63980939eeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:29.240677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b642d-7554-4cee-b055-63980939eeab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:29.241324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b642d-7554-4cee-b055-63980939eeab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:29.241719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b642d-7554-4cee-b055-63980939eeab None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:29.242559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b642d-7554-4cee-b055-63980939eeab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:29.242871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b642d-7554-4cee-b055-63980939eeab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:29.244273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b642d-7554-4cee-b055-63980939eeab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:29.262162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 228/466] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:29.262162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b642d-7554-4cee-b055-63980939eeab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:29.304850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ce7c5-c66d-4298-93af-9c3289ca600e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:29.346595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc064db-b2bc-4d09-893e-94d3fb5a220b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:29.347229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc064db-b2bc-4d09-893e-94d3fb5a220b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:29.347640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc064db-b2bc-4d09-893e-94d3fb5a220b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:29.347906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc064db-b2bc-4d09-893e-94d3fb5a220b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:29.360864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc064db-b2bc-4d09-893e-94d3fb5a220b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:29.361284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:29.363503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc064db-b2bc-4d09-893e-94d3fb5a220b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:29.369842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b642d-7554-4cee-b055-63980939eeab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7bc064db-b2bc-4d09-893e-94d3fb5a220b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:29.379486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:29.380514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:29.381601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:29.381601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:29.381601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:29.381601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 239/467] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:29.381601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc064db-b2bc-4d09-893e-94d3fb5a220b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:29.415569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fda40-65d3-41ae-ba1b-0f9ef1e6d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:29.416237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fda40-65d3-41ae-ba1b-0f9ef1e6d1be None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:29.416452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fda40-65d3-41ae-ba1b-0f9ef1e6d1be None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:29.416716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fda40-65d3-41ae-ba1b-0f9ef1e6d1be None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:29.417367 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fda40-65d3-41ae-ba1b-0f9ef1e6d1be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:29.417703 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fda40-65d3-41ae-ba1b-0f9ef1e6d1be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:29.419086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fda40-65d3-41ae-ba1b-0f9ef1e6d1be None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:29.431615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 229/468] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:29.435665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fda40-65d3-41ae-ba1b-0f9ef1e6d1be None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:29.448518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:29.514784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc064db-b2bc-4d09-893e-94d3fb5a220b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:29.555683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66af7ee-4955-4d22-8755-4297cb0b1362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:29.556672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66af7ee-4955-4d22-8755-4297cb0b1362 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:29.557155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66af7ee-4955-4d22-8755-4297cb0b1362 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:29.557647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66af7ee-4955-4d22-8755-4297cb0b1362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:29.559127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fda40-65d3-41ae-ba1b-0f9ef1e6d1be None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:29.574075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66af7ee-4955-4d22-8755-4297cb0b1362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:29.575718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66af7ee-4955-4d22-8755-4297cb0b1362 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d66af7ee-4955-4d22-8755-4297cb0b1362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:29.592499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:29.593622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:29.594491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:29.594491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:29.594491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:29.594491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 240/469] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:29.594722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66af7ee-4955-4d22-8755-4297cb0b1362 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:29.601126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff50aa-bef2-4173-b7a9-673d62c9aaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:29.601770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff50aa-bef2-4173-b7a9-673d62c9aaff None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:29.602052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff50aa-bef2-4173-b7a9-673d62c9aaff None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:29.602327 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff50aa-bef2-4173-b7a9-673d62c9aaff None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:29.602982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff50aa-bef2-4173-b7a9-673d62c9aaff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:29.604435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff50aa-bef2-4173-b7a9-673d62c9aaff None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:29.625820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 230/470] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:23:29 2022] POST /identity/v3/users => generated 247 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:23:29.626550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff50aa-bef2-4173-b7a9-673d62c9aaff None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:29.642649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:29.742264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66af7ee-4955-4d22-8755-4297cb0b1362 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:29.745548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:29.761413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff50aa-bef2-4173-b7a9-673d62c9aaff None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:29.786692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc5415-20b9-41a0-b2a9-c825fb2758dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:29.787297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc5415-20b9-41a0-b2a9-c825fb2758dc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:29.787754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc5415-20b9-41a0-b2a9-c825fb2758dc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:29.788717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc5415-20b9-41a0-b2a9-c825fb2758dc None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:29.789438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc5415-20b9-41a0-b2a9-c825fb2758dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:29.789792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc5415-20b9-41a0-b2a9-c825fb2758dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:29.791199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc5415-20b9-41a0-b2a9-c825fb2758dc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:29.804631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 241/471] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:29.805973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5217e0-3b7b-4936-a380-262d53df4b52 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:29.806262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5217e0-3b7b-4936-a380-262d53df4b52 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:29.806535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5217e0-3b7b-4936-a380-262d53df4b52 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:29.807802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 242/472] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:29.812538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5217e0-3b7b-4936-a380-262d53df4b52 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:29.816325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502d314-199a-4a95-99e1-90552639f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:29.818476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502d314-199a-4a95-99e1-90552639f5f7 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:29.819328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502d314-199a-4a95-99e1-90552639f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:29.819607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502d314-199a-4a95-99e1-90552639f5f7 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:29.820287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502d314-199a-4a95-99e1-90552639f5f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:29.821788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502d314-199a-4a95-99e1-90552639f5f7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:29.854648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 231/473] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:23:29 2022] POST /identity/v3/users => generated 249 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:23:29.855115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502d314-199a-4a95-99e1-90552639f5f7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:29.966514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:29.970305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5217e0-3b7b-4936-a380-262d53df4b52 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:30.012690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502d314-199a-4a95-99e1-90552639f5f7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:30.023157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1fd4a-6f01-4124-b2ec-5fc8687b96c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:30.024469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1fd4a-6f01-4124-b2ec-5fc8687b96c9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:30.024779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1fd4a-6f01-4124-b2ec-5fc8687b96c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:30.025085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1fd4a-6f01-4124-b2ec-5fc8687b96c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:30.026717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1fd4a-6f01-4124-b2ec-5fc8687b96c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:30.027041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1fd4a-6f01-4124-b2ec-5fc8687b96c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:30.028429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1fd4a-6f01-4124-b2ec-5fc8687b96c9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:30.042792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 243/474] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:30.044160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ae080-e957-4b79-af2e-269c61608831 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:30.045043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ae080-e957-4b79-af2e-269c61608831 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:30.045339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ae080-e957-4b79-af2e-269c61608831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:30.063460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269ff46-d73c-4eeb-b848-d6fb33605216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:30.064443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269ff46-d73c-4eeb-b848-d6fb33605216 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:30.064727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269ff46-d73c-4eeb-b848-d6fb33605216 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:30.065002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269ff46-d73c-4eeb-b848-d6fb33605216 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:30.065702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269ff46-d73c-4eeb-b848-d6fb33605216 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:30.067172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269ff46-d73c-4eeb-b848-d6fb33605216 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:30.069392 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-da5ae080-e957-4b79-af2e-269c61608831 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:30.070753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-da5ae080-e957-4b79-af2e-269c61608831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:30.095495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 232/475] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:23:29 2022] POST /identity/v3/users => generated 249 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:23:30.096291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:30.096415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269ff46-d73c-4eeb-b848-d6fb33605216 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:30.168252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 244/476] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:30.249314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269ff46-d73c-4eeb-b848-d6fb33605216 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:30.283765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:30.301248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb017600-3322-4a4e-b995-b27095f4fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:30.301947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb017600-3322-4a4e-b995-b27095f4fd8d None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:30.302229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb017600-3322-4a4e-b995-b27095f4fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:30.302504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb017600-3322-4a4e-b995-b27095f4fd8d None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:30.303195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb017600-3322-4a4e-b995-b27095f4fd8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:30.304668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb017600-3322-4a4e-b995-b27095f4fd8d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:30.333856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 233/477] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:23:30 2022] POST /identity/v3/users => generated 250 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:23:30.514576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ae080-e957-4b79-af2e-269c61608831 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:30.633066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ae080-e957-4b79-af2e-269c61608831 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:30.677777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:30.678436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:30.678716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:30.678989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:30.679687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:30.681083 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:30.690326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.sql.core [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 17:23:30.690326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 17:23:30.690326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [parameters: {'id': '1b9644b11e9c4ad0ab699d24d36e7b25', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 17:23:30.690326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81532) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Sep 01 17:23:30.708659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:23:30.710545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application yield resource Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise self.value Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application session.commit() Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 17:23:30.711474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.flush() Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application compat.raise_( Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 17:23:30.712350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:23:30.713226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:23:30.715539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [parameters: {'id': '1b9644b11e9c4ad0ab699d24d36e7b25', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:30.717467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 17:23:30.719613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:30.721562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 245/478] 10.222.0.100 () {60 vars in 1136 bytes} [Thu Sep 1 17:23:30 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:30.729116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb017600-3322-4a4e-b995-b27095f4fd8d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:30.876259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb017600-3322-4a4e-b995-b27095f4fd8d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:30.942286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60e81d-2599-4818-bf7e-8930e77199d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:30.943078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60e81d-2599-4818-bf7e-8930e77199d0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:30.943380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60e81d-2599-4818-bf7e-8930e77199d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:30.943693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60e81d-2599-4818-bf7e-8930e77199d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:30.966864 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60e81d-2599-4818-bf7e-8930e77199d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5a60e81d-2599-4818-bf7e-8930e77199d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:30.972508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:30.973476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:23:30.974333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:30.977288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 234/479] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:23:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:30.982556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:31.020638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:31.163096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7557c-9655-4c94-ba6a-7304e6edfe7e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:31.217203 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09406692-1fc7-4cfc-ac52-9a4e55877019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:31.217847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09406692-1fc7-4cfc-ac52-9a4e55877019 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:31.218121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09406692-1fc7-4cfc-ac52-9a4e55877019 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:31.218381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09406692-1fc7-4cfc-ac52-9a4e55877019 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:31.219055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09406692-1fc7-4cfc-ac52-9a4e55877019 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:31.219352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09406692-1fc7-4cfc-ac52-9a4e55877019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:31.220733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09406692-1fc7-4cfc-ac52-9a4e55877019 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:31.230607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 246/480] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:30 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:32.079364 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a06d1-ea07-41a4-ac55-d9487482db16 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:32.080362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a06d1-ea07-41a4-ac55-d9487482db16 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:32.080861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a06d1-ea07-41a4-ac55-d9487482db16 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:32.082725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 235/481] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:32.093458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b7063-781b-43f3-9c16-f8a1eb79363f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:32.094422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b7063-781b-43f3-9c16-f8a1eb79363f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:32.094941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b7063-781b-43f3-9c16-f8a1eb79363f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:32.122625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7f9b7063-781b-43f3-9c16-f8a1eb79363f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:32.124157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7f9b7063-781b-43f3-9c16-f8a1eb79363f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:32.216649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 247/482] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:32.329055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c4f29-3f51-467b-a6f4-17f3d028685e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:32.329451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c4f29-3f51-467b-a6f4-17f3d028685e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:32.329661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c4f29-3f51-467b-a6f4-17f3d028685e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:32.331278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 236/483] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:32.337249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e12e-d724-4753-9168-bead273df862 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:32.337782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e12e-d724-4753-9168-bead273df862 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:32.338256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e12e-d724-4753-9168-bead273df862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:32.367124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ae98e12e-d724-4753-9168-bead273df862 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:32.368689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ae98e12e-d724-4753-9168-bead273df862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:32.372177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:32.464402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 248/484] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:32.565916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c4f29-3f51-467b-a6f4-17f3d028685e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:32.627319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:32.710423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c4f29-3f51-467b-a6f4-17f3d028685e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:32.746591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defccbe-5387-45de-b86c-d9306b11b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:32.747224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defccbe-5387-45de-b86c-d9306b11b76d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:32.747499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defccbe-5387-45de-b86c-d9306b11b76d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:32.747769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defccbe-5387-45de-b86c-d9306b11b76d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:32.759570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defccbe-5387-45de-b86c-d9306b11b76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2defccbe-5387-45de-b86c-d9306b11b76d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:32.764724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:32.766900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:32.768191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 237/485] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:32.774708 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e12e-d724-4753-9168-bead273df862 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:32.899906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e12e-d724-4753-9168-bead273df862 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:32.946393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face6715-d6fe-4ff1-aabe-33cda174b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:32.947270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face6715-d6fe-4ff1-aabe-33cda174b25c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:32.947690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face6715-d6fe-4ff1-aabe-33cda174b25c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:32.948130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face6715-d6fe-4ff1-aabe-33cda174b25c None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:32.948948 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face6715-d6fe-4ff1-aabe-33cda174b25c None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:32.949392 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face6715-d6fe-4ff1-aabe-33cda174b25c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:32.950940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face6715-d6fe-4ff1-aabe-33cda174b25c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:32.960839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 249/486] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:32 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:32.966624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defccbe-5387-45de-b86c-d9306b11b76d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:32.969443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face6715-d6fe-4ff1-aabe-33cda174b25c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:32.973061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:33.088313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:33.090954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face6715-d6fe-4ff1-aabe-33cda174b25c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:33.093252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defccbe-5387-45de-b86c-d9306b11b76d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:33.135194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514e5d8-72ae-417c-8ee9-44302325aa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:33.136035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514e5d8-72ae-417c-8ee9-44302325aa0f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.136375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514e5d8-72ae-417c-8ee9-44302325aa0f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.136748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514e5d8-72ae-417c-8ee9-44302325aa0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.141322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984687d-0165-4e34-aced-396fb8660eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:33.143070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984687d-0165-4e34-aced-396fb8660eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.143743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984687d-0165-4e34-aced-396fb8660eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.144484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984687d-0165-4e34-aced-396fb8660eb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.149845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514e5d8-72ae-417c-8ee9-44302325aa0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:33.151807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514e5d8-72ae-417c-8ee9-44302325aa0f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:33.160067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984687d-0165-4e34-aced-396fb8660eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3984687d-0165-4e34-aced-396fb8660eb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:33.166494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:33.167463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:33.168299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1514e5d8-72ae-417c-8ee9-44302325aa0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:33.168932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:33.169885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:33.171214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:33.171214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:33.171214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:33.171214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 238/487] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:33.172568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86baafd-efa5-4699-9d0b-0c10582c3afc None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.172929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86baafd-efa5-4699-9d0b-0c10582c3afc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.173310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86baafd-efa5-4699-9d0b-0c10582c3afc None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.173721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 250/488] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:33.175771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984687d-0165-4e34-aced-396fb8660eb5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:33.179887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 239/489] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:33 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:33.182166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86baafd-efa5-4699-9d0b-0c10582c3afc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:33.253620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:33.299879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86baafd-efa5-4699-9d0b-0c10582c3afc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:33.305050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984687d-0165-4e34-aced-396fb8660eb5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:33.339179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108d775-6ab0-4d9b-ad21-1ae99d3ff009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:33.340006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108d775-6ab0-4d9b-ad21-1ae99d3ff009 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.340288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108d775-6ab0-4d9b-ad21-1ae99d3ff009 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.340564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108d775-6ab0-4d9b-ad21-1ae99d3ff009 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.341229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108d775-6ab0-4d9b-ad21-1ae99d3ff009 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:33.341530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108d775-6ab0-4d9b-ad21-1ae99d3ff009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:33.342958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108d775-6ab0-4d9b-ad21-1ae99d3ff009 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:33.344729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301ada1-fcd7-44db-8a15-c4afe7e3b071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:33.345345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301ada1-fcd7-44db-8a15-c4afe7e3b071 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.345625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301ada1-fcd7-44db-8a15-c4afe7e3b071 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.345902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301ada1-fcd7-44db-8a15-c4afe7e3b071 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.346558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301ada1-fcd7-44db-8a15-c4afe7e3b071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:33.346864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301ada1-fcd7-44db-8a15-c4afe7e3b071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:33.348232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301ada1-fcd7-44db-8a15-c4afe7e3b071 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:33.353437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 240/490] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:33 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:33.359876 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b957c42-4d35-4eea-bb82-30faa9a1cfae None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.360155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b957c42-4d35-4eea-bb82-30faa9a1cfae None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.360427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b957c42-4d35-4eea-bb82-30faa9a1cfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.361541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 251/491] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:33.370401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55456-dfbe-42de-a5b8-f25f20e90596 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.370401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55456-dfbe-42de-a5b8-f25f20e90596 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.370401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb55456-dfbe-42de-a5b8-f25f20e90596 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.370401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 252/492] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:33 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:33.370926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb55456-dfbe-42de-a5b8-f25f20e90596 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:33.382763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5b957c42-4d35-4eea-bb82-30faa9a1cfae None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:33.384331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5b957c42-4d35-4eea-bb82-30faa9a1cfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:33.396261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:33.478114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 241/493] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:33.479280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320542a3-e04a-4ddb-b83d-c7dad77414e5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.479566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320542a3-e04a-4ddb-b83d-c7dad77414e5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.479834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320542a3-e04a-4ddb-b83d-c7dad77414e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.488148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb55456-dfbe-42de-a5b8-f25f20e90596 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:33.506585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-320542a3-e04a-4ddb-b83d-c7dad77414e5 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:33.507859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-320542a3-e04a-4ddb-b83d-c7dad77414e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:33.521400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:33.541429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156f5f6-8226-44eb-bf99-f193dfb9a5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:33.542828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156f5f6-8226-44eb-bf99-f193dfb9a5ee None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.543441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156f5f6-8226-44eb-bf99-f193dfb9a5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.543991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156f5f6-8226-44eb-bf99-f193dfb9a5ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.559626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156f5f6-8226-44eb-bf99-f193dfb9a5ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:33.561447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156f5f6-8226-44eb-bf99-f193dfb9a5ee None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9156f5f6-8226-44eb-bf99-f193dfb9a5ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:33.580650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:33.581681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:33.582540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:33.582540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:33.582540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:33.583257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 253/494] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:33.584072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156f5f6-8226-44eb-bf99-f193dfb9a5ee None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:33.622002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 242/495] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:33.623320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320542a3-e04a-4ddb-b83d-c7dad77414e5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:33.721285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:33.742539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156f5f6-8226-44eb-bf99-f193dfb9a5ee None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:33.770144 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320542a3-e04a-4ddb-b83d-c7dad77414e5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:33.826093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ff9ec-c17b-422e-b652-0d463e8f39d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:33.826743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ff9ec-c17b-422e-b652-0d463e8f39d1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.827029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ff9ec-c17b-422e-b652-0d463e8f39d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.827304 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ff9ec-c17b-422e-b652-0d463e8f39d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.833618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee93a39-deba-4389-981b-b459d030b173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:33.834263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee93a39-deba-4389-981b-b459d030b173 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:33.834544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee93a39-deba-4389-981b-b459d030b173 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:33.835490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee93a39-deba-4389-981b-b459d030b173 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:33.835917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ff9ec-c17b-422e-b652-0d463e8f39d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:33.836154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee93a39-deba-4389-981b-b459d030b173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:33.836457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee93a39-deba-4389-981b-b459d030b173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:33.837413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ff9ec-c17b-422e-b652-0d463e8f39d1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:33.837972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee93a39-deba-4389-981b-b459d030b173 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-770ff9ec-c17b-422e-b652-0d463e8f39d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:33.851388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:33.852531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:23:33.853502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:33.857494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 254/496] 10.222.0.100 () {56 vars in 1098 bytes} [Thu Sep 1 17:23:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:33.858487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ff9ec-c17b-422e-b652-0d463e8f39d1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:33.858635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 243/497] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:33.859682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee93a39-deba-4389-981b-b459d030b173 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:33.908002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:33.990672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ff9ec-c17b-422e-b652-0d463e8f39d1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:33.998709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee93a39-deba-4389-981b-b459d030b173 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:34.036737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:34.042392 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c12173-bd15-4335-8abb-8ad84324b386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:34.042921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c12173-bd15-4335-8abb-8ad84324b386 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.043206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c12173-bd15-4335-8abb-8ad84324b386 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.043481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c12173-bd15-4335-8abb-8ad84324b386 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.045167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f2d0f-4636-402b-b99b-3112b34607a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:34.046225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f2d0f-4636-402b-b99b-3112b34607a0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.046556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f2d0f-4636-402b-b99b-3112b34607a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.047130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f2d0f-4636-402b-b99b-3112b34607a0 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.047522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f2d0f-4636-402b-b99b-3112b34607a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:34.047836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f2d0f-4636-402b-b99b-3112b34607a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:34.049332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f2d0f-4636-402b-b99b-3112b34607a0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:34.061677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c12173-bd15-4335-8abb-8ad84324b386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-18c12173-bd15-4335-8abb-8ad84324b386 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:34.070704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:34.072408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 255/498] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:34.074776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c12173-bd15-4335-8abb-8ad84324b386 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:34.109250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 244/499] 10.222.0.100 () {56 vars in 1138 bytes} [Thu Sep 1 17:23:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:34.111532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f2d0f-4636-402b-b99b-3112b34607a0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:34.195225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:34.204346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c12173-bd15-4335-8abb-8ad84324b386 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:34.243931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a229b0a-e91f-4061-be75-960d02b15745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:34.244580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a229b0a-e91f-4061-be75-960d02b15745 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.244856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a229b0a-e91f-4061-be75-960d02b15745 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.245243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a229b0a-e91f-4061-be75-960d02b15745 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.251680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f2d0f-4636-402b-b99b-3112b34607a0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:34.253923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a229b0a-e91f-4061-be75-960d02b15745 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:34.255427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a229b0a-e91f-4061-be75-960d02b15745 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5a229b0a-e91f-4061-be75-960d02b15745 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:34.268179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:34.269338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 256/500] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:34.270844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a229b0a-e91f-4061-be75-960d02b15745 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:34.300084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7619bcb-a9e9-4c8e-acf3-1e6eb8323cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:34.300736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7619bcb-a9e9-4c8e-acf3-1e6eb8323cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.301014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7619bcb-a9e9-4c8e-acf3-1e6eb8323cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.301284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7619bcb-a9e9-4c8e-acf3-1e6eb8323cd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.313895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7619bcb-a9e9-4c8e-acf3-1e6eb8323cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b7619bcb-a9e9-4c8e-acf3-1e6eb8323cd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:34.320660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:34.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:34.322583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 245/501] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:34.323613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72194506-58e0-4f0f-a17b-5c5eea75062e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.323772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72194506-58e0-4f0f-a17b-5c5eea75062e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.324036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72194506-58e0-4f0f-a17b-5c5eea75062e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.326692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 246/502] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:34.328031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72194506-58e0-4f0f-a17b-5c5eea75062e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:34.386689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:34.413188 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a229b0a-e91f-4061-be75-960d02b15745 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:34.446101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72194506-58e0-4f0f-a17b-5c5eea75062e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:34.451579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a56dff-98eb-4c3f-a216-9bf66f9510da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:34.452196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a56dff-98eb-4c3f-a216-9bf66f9510da None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.452475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a56dff-98eb-4c3f-a216-9bf66f9510da None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.452746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a56dff-98eb-4c3f-a216-9bf66f9510da None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.453422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a56dff-98eb-4c3f-a216-9bf66f9510da None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:34.453759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a56dff-98eb-4c3f-a216-9bf66f9510da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:34.455169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a56dff-98eb-4c3f-a216-9bf66f9510da None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:34.464110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 257/503] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:34 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:34.464960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a56dff-98eb-4c3f-a216-9bf66f9510da None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:34.485334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d0110-953b-4bd5-b1ce-616999e3820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:34.485987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0110-953b-4bd5-b1ce-616999e3820a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.486266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0110-953b-4bd5-b1ce-616999e3820a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.486534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0110-953b-4bd5-b1ce-616999e3820a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.488040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:34.563801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d0110-953b-4bd5-b1ce-616999e3820a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:34.565190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d0110-953b-4bd5-b1ce-616999e3820a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:34.591035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a56dff-98eb-4c3f-a216-9bf66f9510da None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:34.619773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:34.632850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f418f6f-9d0b-4638-8782-3a0348a90377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:34.633466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f418f6f-9d0b-4638-8782-3a0348a90377 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.633796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f418f6f-9d0b-4638-8782-3a0348a90377 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.634065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f418f6f-9d0b-4638-8782-3a0348a90377 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.634803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f418f6f-9d0b-4638-8782-3a0348a90377 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:34.635100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f418f6f-9d0b-4638-8782-3a0348a90377 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:34.636543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f418f6f-9d0b-4638-8782-3a0348a90377 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:34.676437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 258/504] 10.222.0.100 () {56 vars in 1142 bytes} [Thu Sep 1 17:23:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:34.677300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f418f6f-9d0b-4638-8782-3a0348a90377 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:34.681184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 247/505] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:34 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:34.682774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea40d9-cce3-4cf2-97f1-21626ae83b5a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.683064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea40d9-cce3-4cf2-97f1-21626ae83b5a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.683340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea40d9-cce3-4cf2-97f1-21626ae83b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.703071 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-11ea40d9-cce3-4cf2-97f1-21626ae83b5a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:34.704353 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-11ea40d9-cce3-4cf2-97f1-21626ae83b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:34.786550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:34.788898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 248/506] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:34.789840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea40d9-cce3-4cf2-97f1-21626ae83b5a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:34.823668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f418f6f-9d0b-4638-8782-3a0348a90377 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:34.866601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ed57e-1091-401e-b6eb-ea3b9e418ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:34.867209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ed57e-1091-401e-b6eb-ea3b9e418ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.867497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ed57e-1091-401e-b6eb-ea3b9e418ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.867768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ed57e-1091-401e-b6eb-ea3b9e418ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.868445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ed57e-1091-401e-b6eb-ea3b9e418ea0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:34.868746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ed57e-1091-401e-b6eb-ea3b9e418ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:34.870189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ed57e-1091-401e-b6eb-ea3b9e418ea0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:34.878944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 259/507] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:34 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:34.879620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ed57e-1091-401e-b6eb-ea3b9e418ea0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:34.895845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:34.919804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea40d9-cce3-4cf2-97f1-21626ae83b5a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:34.962256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09556ee0-25c3-4306-8b02-f7b7a975edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:34.962820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09556ee0-25c3-4306-8b02-f7b7a975edef None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:34.963103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09556ee0-25c3-4306-8b02-f7b7a975edef None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:34.963376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09556ee0-25c3-4306-8b02-f7b7a975edef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:34.977238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09556ee0-25c3-4306-8b02-f7b7a975edef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:34.979332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09556ee0-25c3-4306-8b02-f7b7a975edef None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-09556ee0-25c3-4306-8b02-f7b7a975edef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:34.997890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:34.999041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:35.005133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:35.005133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:35.005133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:35.005133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 249/508] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:35.005133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09556ee0-25c3-4306-8b02-f7b7a975edef None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:35.012684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ed57e-1091-401e-b6eb-ea3b9e418ea0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:35.050378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dcf50-5371-41aa-890d-87e5a2aa4ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:35.051929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dcf50-5371-41aa-890d-87e5a2aa4ebc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:35.052217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dcf50-5371-41aa-890d-87e5a2aa4ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:35.052493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dcf50-5371-41aa-890d-87e5a2aa4ebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:35.057400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:35.146063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09556ee0-25c3-4306-8b02-f7b7a975edef None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:35.156770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241dcf50-5371-41aa-890d-87e5a2aa4ebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:35.158369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241dcf50-5371-41aa-890d-87e5a2aa4ebc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:35.182060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:35.188025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d468aa8-4b97-4d01-b011-08b22537c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:35.188667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d468aa8-4b97-4d01-b011-08b22537c16f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:35.188946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d468aa8-4b97-4d01-b011-08b22537c16f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:35.189222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d468aa8-4b97-4d01-b011-08b22537c16f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:35.202701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d468aa8-4b97-4d01-b011-08b22537c16f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:35.204208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d468aa8-4b97-4d01-b011-08b22537c16f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8d468aa8-4b97-4d01-b011-08b22537c16f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:35.221312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:35.223262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:35.225008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:35.225008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:35.225008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:35.225008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 250/509] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:35.225008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d468aa8-4b97-4d01-b011-08b22537c16f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:35.283024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 260/510] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:34 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:35.283903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dcf50-5371-41aa-890d-87e5a2aa4ebc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:35.324023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:35.345571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d468aa8-4b97-4d01-b011-08b22537c16f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:35.391387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dfa57-8bb6-4437-abfe-034680795e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:35.392374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dfa57-8bb6-4437-abfe-034680795e51 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:35.392655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dfa57-8bb6-4437-abfe-034680795e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:35.393050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dfa57-8bb6-4437-abfe-034680795e51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:35.406618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dcf50-5371-41aa-890d-87e5a2aa4ebc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:35.408406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dfa57-8bb6-4437-abfe-034680795e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:35.410760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dfa57-8bb6-4437-abfe-034680795e51 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-546dfa57-8bb6-4437-abfe-034680795e51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:35.434628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:35.435718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:35.441884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:35.441884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:35.441884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:35.441884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 251/511] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:35.441884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dfa57-8bb6-4437-abfe-034680795e51 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:35.453611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb659988-3a5a-4382-9e20-6a4fcbc26b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:35.454215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb659988-3a5a-4382-9e20-6a4fcbc26b9c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:35.454493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb659988-3a5a-4382-9e20-6a4fcbc26b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:35.454761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb659988-3a5a-4382-9e20-6a4fcbc26b9c None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:35.455415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb659988-3a5a-4382-9e20-6a4fcbc26b9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:35.455718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb659988-3a5a-4382-9e20-6a4fcbc26b9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:35.457130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb659988-3a5a-4382-9e20-6a4fcbc26b9c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:35.469759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 261/512] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:35.470626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb659988-3a5a-4382-9e20-6a4fcbc26b9c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:35.491039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:35.578010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dfa57-8bb6-4437-abfe-034680795e51 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:35.580498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:35.590484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb659988-3a5a-4382-9e20-6a4fcbc26b9c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:35.626894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c22b1b-4549-44d9-ba2d-5955656b76aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:35.627543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c22b1b-4549-44d9-ba2d-5955656b76aa None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:35.627823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c22b1b-4549-44d9-ba2d-5955656b76aa None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:35.628097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c22b1b-4549-44d9-ba2d-5955656b76aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:35.638095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c839c-6b75-40fb-95c0-c619be02a3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:35.639136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c839c-6b75-40fb-95c0-c619be02a3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:35.639493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c22b1b-4549-44d9-ba2d-5955656b76aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:35.639591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c839c-6b75-40fb-95c0-c619be02a3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:35.640074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c839c-6b75-40fb-95c0-c619be02a3f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:35.640944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c22b1b-4549-44d9-ba2d-5955656b76aa None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:35.641275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c839c-6b75-40fb-95c0-c619be02a3f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:35.641852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c839c-6b75-40fb-95c0-c619be02a3f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:35.644354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c839c-6b75-40fb-95c0-c619be02a3f0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-37c22b1b-4549-44d9-ba2d-5955656b76aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:35.658170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:35.659283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:35.668105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:35.668105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:35.668105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:35.668105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 252/513] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:35.668105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c22b1b-4549-44d9-ba2d-5955656b76aa None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:35.668382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 262/514] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:35.668382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c839c-6b75-40fb-95c0-c619be02a3f0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:35.774639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c839c-6b75-40fb-95c0-c619be02a3f0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:35.777122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:35.796355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c22b1b-4549-44d9-ba2d-5955656b76aa None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:35.815628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceff3e6-9d76-4cea-8204-dad0b546f289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:35.816648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff3e6-9d76-4cea-8204-dad0b546f289 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:35.817118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff3e6-9d76-4cea-8204-dad0b546f289 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:35.817592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff3e6-9d76-4cea-8204-dad0b546f289 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:35.819230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceff3e6-9d76-4cea-8204-dad0b546f289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:35.819755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceff3e6-9d76-4cea-8204-dad0b546f289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:35.821598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceff3e6-9d76-4cea-8204-dad0b546f289 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:35.838928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 263/515] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:35.838928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceff3e6-9d76-4cea-8204-dad0b546f289 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:35.862823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380d16b-eeb9-4de3-8ca9-af8eef768382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:35.863737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380d16b-eeb9-4de3-8ca9-af8eef768382 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:35.864180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380d16b-eeb9-4de3-8ca9-af8eef768382 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:35.864625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380d16b-eeb9-4de3-8ca9-af8eef768382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:35.869407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:35.880532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380d16b-eeb9-4de3-8ca9-af8eef768382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:35.883078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380d16b-eeb9-4de3-8ca9-af8eef768382 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9380d16b-eeb9-4de3-8ca9-af8eef768382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:35.898556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:35.900324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:35.901840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:35.901840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:35.901840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:35.901840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 253/516] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:35.901840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380d16b-eeb9-4de3-8ca9-af8eef768382 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:35.984116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceff3e6-9d76-4cea-8204-dad0b546f289 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.022146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eaae9-394f-4036-be25-282da49cd495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.022704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eaae9-394f-4036-be25-282da49cd495 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.022898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eaae9-394f-4036-be25-282da49cd495 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.023171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eaae9-394f-4036-be25-282da49cd495 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.023541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380d16b-eeb9-4de3-8ca9-af8eef768382 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.032764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599eaae9-394f-4036-be25-282da49cd495 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.034255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599eaae9-394f-4036-be25-282da49cd495 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-599eaae9-394f-4036-be25-282da49cd495 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:36.046018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:36.047053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 264/517] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:36.047960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eaae9-394f-4036-be25-282da49cd495 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.064331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a028d-32e6-40e6-a7db-e9b3f370af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.065062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a028d-32e6-40e6-a7db-e9b3f370af56 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.065403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a028d-32e6-40e6-a7db-e9b3f370af56 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.065708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a028d-32e6-40e6-a7db-e9b3f370af56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.074623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a028d-32e6-40e6-a7db-e9b3f370af56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.075982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a028d-32e6-40e6-a7db-e9b3f370af56 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.079121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a14a028d-32e6-40e6-a7db-e9b3f370af56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:36.099933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:36.101032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 254/518] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:36.102081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a028d-32e6-40e6-a7db-e9b3f370af56 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.142120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:36.166200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eaae9-394f-4036-be25-282da49cd495 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.200671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b874c44-58a3-4a1d-a9e1-378b3f95e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.201274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b874c44-58a3-4a1d-a9e1-378b3f95e814 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.201549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b874c44-58a3-4a1d-a9e1-378b3f95e814 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.201847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b874c44-58a3-4a1d-a9e1-378b3f95e814 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.202482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b874c44-58a3-4a1d-a9e1-378b3f95e814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:36.202782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b874c44-58a3-4a1d-a9e1-378b3f95e814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.204149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b874c44-58a3-4a1d-a9e1-378b3f95e814 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.215108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a028d-32e6-40e6-a7db-e9b3f370af56 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.215930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 265/519] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:36.216759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b874c44-58a3-4a1d-a9e1-378b3f95e814 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.261756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6fa3f-b90d-4a77-9ecc-89bd764f689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.262363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6fa3f-b90d-4a77-9ecc-89bd764f689f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.262645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6fa3f-b90d-4a77-9ecc-89bd764f689f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.262937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6fa3f-b90d-4a77-9ecc-89bd764f689f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.270783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6fa3f-b90d-4a77-9ecc-89bd764f689f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.272287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6fa3f-b90d-4a77-9ecc-89bd764f689f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-ffb6fa3f-b90d-4a77-9ecc-89bd764f689f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:36.285647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:36.286752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 255/520] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:36.288346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6fa3f-b90d-4a77-9ecc-89bd764f689f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.326359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b874c44-58a3-4a1d-a9e1-378b3f95e814 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.353066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:36.363491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6019cd-1f5f-4c6f-a9bf-f888b0d5653c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.364091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6019cd-1f5f-4c6f-a9bf-f888b0d5653c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.364371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6019cd-1f5f-4c6f-a9bf-f888b0d5653c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.364649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6019cd-1f5f-4c6f-a9bf-f888b0d5653c None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.365290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6019cd-1f5f-4c6f-a9bf-f888b0d5653c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:36.366192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6019cd-1f5f-4c6f-a9bf-f888b0d5653c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.367997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6019cd-1f5f-4c6f-a9bf-f888b0d5653c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.371023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:36.386250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 266/521] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:36.387219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6019cd-1f5f-4c6f-a9bf-f888b0d5653c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.398329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6fa3f-b90d-4a77-9ecc-89bd764f689f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.434788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e703ae0-9a51-40c6-b9e0-2258d2e07d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.435400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e703ae0-9a51-40c6-b9e0-2258d2e07d03 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.435692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e703ae0-9a51-40c6-b9e0-2258d2e07d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.435966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e703ae0-9a51-40c6-b9e0-2258d2e07d03 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.436705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e703ae0-9a51-40c6-b9e0-2258d2e07d03 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:36.437014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e703ae0-9a51-40c6-b9e0-2258d2e07d03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.438513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e703ae0-9a51-40c6-b9e0-2258d2e07d03 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.456391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 256/522] 10.222.0.100 () {56 vars in 1142 bytes} [Thu Sep 1 17:23:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:36.457230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e703ae0-9a51-40c6-b9e0-2258d2e07d03 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.493526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6019cd-1f5f-4c6f-a9bf-f888b0d5653c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.527981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffcd52-2d67-450a-b6b8-3baa3bbafb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.528505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffcd52-2d67-450a-b6b8-3baa3bbafb6c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.528789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffcd52-2d67-450a-b6b8-3baa3bbafb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.529063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffcd52-2d67-450a-b6b8-3baa3bbafb6c None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.529790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffcd52-2d67-450a-b6b8-3baa3bbafb6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:36.530104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffcd52-2d67-450a-b6b8-3baa3bbafb6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.531544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffcd52-2d67-450a-b6b8-3baa3bbafb6c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.550562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 267/523] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:36.551695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffcd52-2d67-450a-b6b8-3baa3bbafb6c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.560964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e703ae0-9a51-40c6-b9e0-2258d2e07d03 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.581177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:36.610544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c422e-0860-4bc5-ad8d-8aa1c593b0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.611594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c422e-0860-4bc5-ad8d-8aa1c593b0fc None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.612102 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c422e-0860-4bc5-ad8d-8aa1c593b0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.612588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c422e-0860-4bc5-ad8d-8aa1c593b0fc None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.613265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:36.614467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c422e-0860-4bc5-ad8d-8aa1c593b0fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.618219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c422e-0860-4bc5-ad8d-8aa1c593b0fc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.652672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 257/524] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:23:36 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:23:36.654188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c422e-0860-4bc5-ad8d-8aa1c593b0fc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.667740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffcd52-2d67-450a-b6b8-3baa3bbafb6c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.715300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd832c8-71f3-4208-b722-8d628c8c2ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.716111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd832c8-71f3-4208-b722-8d628c8c2ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.716505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd832c8-71f3-4208-b722-8d628c8c2ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.716880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd832c8-71f3-4208-b722-8d628c8c2ba7 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.717793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd832c8-71f3-4208-b722-8d628c8c2ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:36.718201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd832c8-71f3-4208-b722-8d628c8c2ba7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.719821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd832c8-71f3-4208-b722-8d628c8c2ba7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.750530 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 268/525] 10.222.0.100 () {56 vars in 1144 bytes} [Thu Sep 1 17:23:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:36.750530 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd832c8-71f3-4208-b722-8d628c8c2ba7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.823628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c422e-0860-4bc5-ad8d-8aa1c593b0fc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.867468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95633b2-5c50-4a36-8bf6-797fb473d556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.868113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95633b2-5c50-4a36-8bf6-797fb473d556 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.868392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95633b2-5c50-4a36-8bf6-797fb473d556 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.868666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95633b2-5c50-4a36-8bf6-797fb473d556 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.871773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd832c8-71f3-4208-b722-8d628c8c2ba7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:36.876641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95633b2-5c50-4a36-8bf6-797fb473d556 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:36.878173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95633b2-5c50-4a36-8bf6-797fb473d556 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:36.889276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c95633b2-5c50-4a36-8bf6-797fb473d556 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:36.894236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:36.895132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:36.895997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:36.895997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:36.895997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:36.895997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:36.895997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:36.895997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:36.895997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:36.895997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:36.895997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 258/526] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:36.896540 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95633b2-5c50-4a36-8bf6-797fb473d556 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:36.918574 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860179a6-1949-413f-bfd5-54b6c48888e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:36.919271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860179a6-1949-413f-bfd5-54b6c48888e8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:36.919476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860179a6-1949-413f-bfd5-54b6c48888e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:36.919733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860179a6-1949-413f-bfd5-54b6c48888e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:36.938656 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:37.002333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860179a6-1949-413f-bfd5-54b6c48888e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:37.003825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860179a6-1949-413f-bfd5-54b6c48888e8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:37.025991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95633b2-5c50-4a36-8bf6-797fb473d556 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:37.077244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d250cd-b5f4-400f-918f-84c63eed9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:37.078229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d250cd-b5f4-400f-918f-84c63eed9a6b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:37.078716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d250cd-b5f4-400f-918f-84c63eed9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:37.079210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d250cd-b5f4-400f-918f-84c63eed9a6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:37.080608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d250cd-b5f4-400f-918f-84c63eed9a6b None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:37.081176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d250cd-b5f4-400f-918f-84c63eed9a6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:37.083984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d250cd-b5f4-400f-918f-84c63eed9a6b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:37.097620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 259/527] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:36 2022] GET /identity/v3/role_assignments?user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:37.097620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d250cd-b5f4-400f-918f-84c63eed9a6b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:37.154100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 269/528] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:36 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:37.155094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860179a6-1949-413f-bfd5-54b6c48888e8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:37.175083 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:37.224879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:37.229889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d250cd-b5f4-400f-918f-84c63eed9a6b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:37.269270 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e537d-0fea-440f-a074-3e4faba74416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:37.269923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e537d-0fea-440f-a074-3e4faba74416 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:37.270232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e537d-0fea-440f-a074-3e4faba74416 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:37.270478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e537d-0fea-440f-a074-3e4faba74416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:37.289222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860179a6-1949-413f-bfd5-54b6c48888e8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:37.337756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661315bd-1e23-4d30-b428-d45266e57e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:37.338372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661315bd-1e23-4d30-b428-d45266e57e6b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:37.338607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661315bd-1e23-4d30-b428-d45266e57e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:37.338881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661315bd-1e23-4d30-b428-d45266e57e6b None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:37.339779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661315bd-1e23-4d30-b428-d45266e57e6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:37.340448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661315bd-1e23-4d30-b428-d45266e57e6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:37.341981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661315bd-1e23-4d30-b428-d45266e57e6b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:37.352107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e537d-0fea-440f-a074-3e4faba74416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:37.354639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e537d-0fea-440f-a074-3e4faba74416 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:37.356238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 270/529] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:37.357299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661315bd-1e23-4d30-b428-d45266e57e6b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:37.449256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:37.480129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661315bd-1e23-4d30-b428-d45266e57e6b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:37.486508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:37.496395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 260/530] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:37.496395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e537d-0fea-440f-a074-3e4faba74416 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:37.533055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f11dd1-cfbc-4b19-b376-7e43a80e2396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:37.534197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f11dd1-cfbc-4b19-b376-7e43a80e2396 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:37.534693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f11dd1-cfbc-4b19-b376-7e43a80e2396 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:37.535157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f11dd1-cfbc-4b19-b376-7e43a80e2396 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:37.548744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f11dd1-cfbc-4b19-b376-7e43a80e2396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:37.550613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f11dd1-cfbc-4b19-b376-7e43a80e2396 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-79f11dd1-cfbc-4b19-b376-7e43a80e2396 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:37.573944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:37.577128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:37.578816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:37.578816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:37.578816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:37.578816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 271/531] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:37.578816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f11dd1-cfbc-4b19-b376-7e43a80e2396 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:37.667343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e537d-0fea-440f-a074-3e4faba74416 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:37.715072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bfde2-6635-41b2-bd69-1e617be83290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:37.715677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bfde2-6635-41b2-bd69-1e617be83290 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:37.715955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bfde2-6635-41b2-bd69-1e617be83290 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:37.716223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bfde2-6635-41b2-bd69-1e617be83290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:37.717054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bfde2-6635-41b2-bd69-1e617be83290 None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=e0891d81e6e24ad99a9b532059c8c2af) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:37.717362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bfde2-6635-41b2-bd69-1e617be83290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:37.718940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bfde2-6635-41b2-bd69-1e617be83290 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:37.723516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f11dd1-cfbc-4b19-b376-7e43a80e2396 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:37.729900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 261/532] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:37 2022] GET /identity/v3/role_assignments?user.id=e0891d81e6e24ad99a9b532059c8c2af&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:37.730924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bfde2-6635-41b2-bd69-1e617be83290 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:37.758943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:37.783549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02ef36-2beb-4d4c-87db-784181c350c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:37.784208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ef36-2beb-4d4c-87db-784181c350c7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:37.784495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ef36-2beb-4d4c-87db-784181c350c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:37.784835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ef36-2beb-4d4c-87db-784181c350c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:37.808675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02ef36-2beb-4d4c-87db-784181c350c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:37.810669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02ef36-2beb-4d4c-87db-784181c350c7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:37.830771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-fd02ef36-2beb-4d4c-87db-784181c350c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:37.842180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:37.843426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:37.844380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:37.844380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:37.844380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:37.844380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 272/533] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:37.849717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02ef36-2beb-4d4c-87db-784181c350c7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:37.885647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bfde2-6635-41b2-bd69-1e617be83290 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:37.928451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf82286-8aec-47b3-ab75-d092fbb818b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:37.930075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf82286-8aec-47b3-ab75-d092fbb818b1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:37.930531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf82286-8aec-47b3-ab75-d092fbb818b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:37.930954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf82286-8aec-47b3-ab75-d092fbb818b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:37.931785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf82286-8aec-47b3-ab75-d092fbb818b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:37.932234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf82286-8aec-47b3-ab75-d092fbb818b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:37.934192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf82286-8aec-47b3-ab75-d092fbb818b1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:37.951580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 262/534] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:37.960124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf82286-8aec-47b3-ab75-d092fbb818b1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:38.030531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02ef36-2beb-4d4c-87db-784181c350c7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:38.069811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb423f4e-1487-4af4-9684-a1f60d48e064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:38.070422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb423f4e-1487-4af4-9684-a1f60d48e064 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:38.070705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb423f4e-1487-4af4-9684-a1f60d48e064 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:38.070980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb423f4e-1487-4af4-9684-a1f60d48e064 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:38.071639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb423f4e-1487-4af4-9684-a1f60d48e064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:38.071943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb423f4e-1487-4af4-9684-a1f60d48e064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:38.073333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb423f4e-1487-4af4-9684-a1f60d48e064 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:38.088357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 273/535] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:38.096639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb423f4e-1487-4af4-9684-a1f60d48e064 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:38.124091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf82286-8aec-47b3-ab75-d092fbb818b1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:38.161996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:38.172197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd107-170d-459b-852f-190e2981db1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:38.173549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd107-170d-459b-852f-190e2981db1a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:38.174896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd107-170d-459b-852f-190e2981db1a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:38.175405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd107-170d-459b-852f-190e2981db1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:38.186385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cd107-170d-459b-852f-190e2981db1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:38.188508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cd107-170d-459b-852f-190e2981db1a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a79cd107-170d-459b-852f-190e2981db1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:38.203010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:38.204111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:38.204948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:38.204948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:38.204948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:38.204948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:38.204948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:38.204948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:38.204948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:38.204948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:38.207897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 263/536] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:38.214261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd107-170d-459b-852f-190e2981db1a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:38.221941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:38.293515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb423f4e-1487-4af4-9684-a1f60d48e064 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:38.344515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd107-170d-459b-852f-190e2981db1a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:38.364654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e233ab-3c62-4a81-a7ac-e186ce787924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:38.365319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e233ab-3c62-4a81-a7ac-e186ce787924 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:38.365659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e233ab-3c62-4a81-a7ac-e186ce787924 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:38.365999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e233ab-3c62-4a81-a7ac-e186ce787924 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:38.385791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e233ab-3c62-4a81-a7ac-e186ce787924 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:38.387438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e233ab-3c62-4a81-a7ac-e186ce787924 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:38.401215 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc65972-e2df-4bba-8018-f7d33f867d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:38.402509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc65972-e2df-4bba-8018-f7d33f867d6e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:38.402985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc65972-e2df-4bba-8018-f7d33f867d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:38.403473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc65972-e2df-4bba-8018-f7d33f867d6e None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:38.404395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc65972-e2df-4bba-8018-f7d33f867d6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:38.404896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc65972-e2df-4bba-8018-f7d33f867d6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:38.406639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc65972-e2df-4bba-8018-f7d33f867d6e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f7e233ab-3c62-4a81-a7ac-e186ce787924 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:38.412107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:38.414133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:38.414985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:38.414985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:38.414985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:38.414985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:38.414985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:38.414985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:38.414985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:38.414985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:38.418866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 274/537] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:38.442254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e233ab-3c62-4a81-a7ac-e186ce787924 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:38.461603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 264/538] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:38.469488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc65972-e2df-4bba-8018-f7d33f867d6e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:38.591504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:38.646298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:38.652037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e233ab-3c62-4a81-a7ac-e186ce787924 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:38.698593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eba2cc-ff1e-4000-ad8b-2c0cc3a71975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:38.699323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eba2cc-ff1e-4000-ad8b-2c0cc3a71975 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:38.699607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eba2cc-ff1e-4000-ad8b-2c0cc3a71975 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:38.699886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eba2cc-ff1e-4000-ad8b-2c0cc3a71975 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:38.700647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eba2cc-ff1e-4000-ad8b-2c0cc3a71975 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:38.700955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eba2cc-ff1e-4000-ad8b-2c0cc3a71975 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:38.702794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eba2cc-ff1e-4000-ad8b-2c0cc3a71975 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:38.729992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 275/539] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:38.739194 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eba2cc-ff1e-4000-ad8b-2c0cc3a71975 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:38.753876 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc65972-e2df-4bba-8018-f7d33f867d6e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:38.858369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03227a-6e7c-4e7a-bc5d-ab298bbb2279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:38.859581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03227a-6e7c-4e7a-bc5d-ab298bbb2279 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:38.860075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03227a-6e7c-4e7a-bc5d-ab298bbb2279 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:38.860610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03227a-6e7c-4e7a-bc5d-ab298bbb2279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:38.861726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03227a-6e7c-4e7a-bc5d-ab298bbb2279 None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=1d47d8e089f446d3bb4b6609d7256ac2) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:38.862270 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03227a-6e7c-4e7a-bc5d-ab298bbb2279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:38.864047 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03227a-6e7c-4e7a-bc5d-ab298bbb2279 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:38.884380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 265/540] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:38 2022] GET /identity/v3/role_assignments?user.id=1d47d8e089f446d3bb4b6609d7256ac2&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 260 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:38.993075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eba2cc-ff1e-4000-ad8b-2c0cc3a71975 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:39.011491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463bb8a-627a-4683-9d88-fcf138d21e24 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:39.012129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463bb8a-627a-4683-9d88-fcf138d21e24 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:39.012606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463bb8a-627a-4683-9d88-fcf138d21e24 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:39.014653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 266/541] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:39 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:39.020816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda88f-3ce2-470d-84cc-29259034e75f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:39.021093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda88f-3ce2-470d-84cc-29259034e75f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:39.021358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda88f-3ce2-470d-84cc-29259034e75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:39.069589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a6eda88f-3ce2-470d-84cc-29259034e75f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:39.073683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a6eda88f-3ce2-470d-84cc-29259034e75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:39.111579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799499-cdcb-4c20-9c1b-e99b74028e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:39.112631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799499-cdcb-4c20-9c1b-e99b74028e97 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:39.113128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799499-cdcb-4c20-9c1b-e99b74028e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:39.113641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799499-cdcb-4c20-9c1b-e99b74028e97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:39.115053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48799499-cdcb-4c20-9c1b-e99b74028e97 None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=bbe180b45f2e49e2abbf6e833ce0d44c) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:39.116159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48799499-cdcb-4c20-9c1b-e99b74028e97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:39.118943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48799499-cdcb-4c20-9c1b-e99b74028e97 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:39.130309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 276/542] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:38 2022] GET /identity/v3/role_assignments?user.id=bbe180b45f2e49e2abbf6e833ce0d44c&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 260 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:39.246149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 267/543] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:39.382420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6554-d6bc-4a3b-a75b-972b49361c37 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:39.382952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6554-d6bc-4a3b-a75b-972b49361c37 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:39.382952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6554-d6bc-4a3b-a75b-972b49361c37 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:39.384635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 277/544] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:39 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:39.390071 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a945231-8a7b-4b8c-829f-bd626bd7fdaa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:39.390605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a945231-8a7b-4b8c-829f-bd626bd7fdaa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:39.390605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a945231-8a7b-4b8c-829f-bd626bd7fdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:39.418471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0a945231-8a7b-4b8c-829f-bd626bd7fdaa None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:39.420143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0a945231-8a7b-4b8c-829f-bd626bd7fdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:39.461028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:39.514541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 268/545] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:23:39.658343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd6554-d6bc-4a3b-a75b-972b49361c37 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:39.687968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:39.800826 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd6554-d6bc-4a3b-a75b-972b49361c37 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:39.840793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f61e2-53d1-40d1-8300-61e27a89ba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:39.841439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f61e2-53d1-40d1-8300-61e27a89ba9f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:39.841759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f61e2-53d1-40d1-8300-61e27a89ba9f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:39.842037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f61e2-53d1-40d1-8300-61e27a89ba9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:39.854601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f61e2-53d1-40d1-8300-61e27a89ba9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-2d0f61e2-53d1-40d1-8300-61e27a89ba9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:39.859674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:39.860683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:39.861744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 278/546] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:39.869545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a945231-8a7b-4b8c-829f-bd626bd7fdaa None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:39.886816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f61e2-53d1-40d1-8300-61e27a89ba9f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:40.049249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a945231-8a7b-4b8c-829f-bd626bd7fdaa None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:40.071703 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:40.103490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f61e2-53d1-40d1-8300-61e27a89ba9f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:40.131495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394f973-f635-4a5c-afa0-351ef30594ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:40.132168 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394f973-f635-4a5c-afa0-351ef30594ca None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:40.132483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394f973-f635-4a5c-afa0-351ef30594ca None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:40.134758 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394f973-f635-4a5c-afa0-351ef30594ca None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:40.135496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394f973-f635-4a5c-afa0-351ef30594ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:40.135802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394f973-f635-4a5c-afa0-351ef30594ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:40.138354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394f973-f635-4a5c-afa0-351ef30594ca None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:40.156744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 269/547] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:39 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:40.162172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3a550-2428-4d1e-8804-f4881e2dc642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:40.165029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394f973-f635-4a5c-afa0-351ef30594ca None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:40.168444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3a550-2428-4d1e-8804-f4881e2dc642 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:40.168719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3a550-2428-4d1e-8804-f4881e2dc642 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:40.168992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3a550-2428-4d1e-8804-f4881e2dc642 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:40.192373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3a550-2428-4d1e-8804-f4881e2dc642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a1d3a550-2428-4d1e-8804-f4881e2dc642 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:40.199385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:40.200374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:40.201179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:40.202629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 279/548] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:40.211247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3a550-2428-4d1e-8804-f4881e2dc642 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:40.229036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:40.413740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394f973-f635-4a5c-afa0-351ef30594ca None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:40.470288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a5adc-f936-4f1f-8c9b-7ac05f5a3a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:40.471089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a5adc-f936-4f1f-8c9b-7ac05f5a3a8c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:40.471375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a5adc-f936-4f1f-8c9b-7ac05f5a3a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:40.471646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a5adc-f936-4f1f-8c9b-7ac05f5a3a8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:40.506009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a5adc-f936-4f1f-8c9b-7ac05f5a3a8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:40.507749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3a550-2428-4d1e-8804-f4881e2dc642 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:40.508377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a5adc-f936-4f1f-8c9b-7ac05f5a3a8c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-da3a5adc-f936-4f1f-8c9b-7ac05f5a3a8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:40.544792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:40.546088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:40.547237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:40.547237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:40.547237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:40.547237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 270/549] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:40.554646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a5adc-f936-4f1f-8c9b-7ac05f5a3a8c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:40.610219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97550802-fc0b-40fd-aac5-3aa11f98f499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:40.611485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97550802-fc0b-40fd-aac5-3aa11f98f499 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:40.611834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97550802-fc0b-40fd-aac5-3aa11f98f499 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:40.612236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97550802-fc0b-40fd-aac5-3aa11f98f499 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:40.613247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97550802-fc0b-40fd-aac5-3aa11f98f499 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:40.613701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97550802-fc0b-40fd-aac5-3aa11f98f499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:40.617148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97550802-fc0b-40fd-aac5-3aa11f98f499 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:40.633192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 280/550] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:40 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:40.634936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d6e43-858e-405c-bfe6-7e46c055a92b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:40.635382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d6e43-858e-405c-bfe6-7e46c055a92b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:40.635818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d6e43-858e-405c-bfe6-7e46c055a92b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:40.637659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 281/551] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:40.639549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d6e43-858e-405c-bfe6-7e46c055a92b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:40.644047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:40.705762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:40.738280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a5adc-f936-4f1f-8c9b-7ac05f5a3a8c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:40.789971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a241a-0380-4d3d-97ac-b6fb27ae3a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:40.790777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a241a-0380-4d3d-97ac-b6fb27ae3a0e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:40.791191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a241a-0380-4d3d-97ac-b6fb27ae3a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:40.791843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a241a-0380-4d3d-97ac-b6fb27ae3a0e None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:40.792632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a241a-0380-4d3d-97ac-b6fb27ae3a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:40.793041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a241a-0380-4d3d-97ac-b6fb27ae3a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:40.795482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a241a-0380-4d3d-97ac-b6fb27ae3a0e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:40.800249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d6e43-858e-405c-bfe6-7e46c055a92b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:40.814066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 271/552] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:40.816885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588472c6-a7c8-4e47-bd10-0e6ef4919f21 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:40.817728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588472c6-a7c8-4e47-bd10-0e6ef4919f21 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:40.818504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588472c6-a7c8-4e47-bd10-0e6ef4919f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:40.847842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e2ee-37b4-4d3f-bbd9-3f2746bf3557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:40.848639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e2ee-37b4-4d3f-bbd9-3f2746bf3557 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:40.849022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e2ee-37b4-4d3f-bbd9-3f2746bf3557 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:40.849398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e2ee-37b4-4d3f-bbd9-3f2746bf3557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:40.851520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-588472c6-a7c8-4e47-bd10-0e6ef4919f21 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:40.853433 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-588472c6-a7c8-4e47-bd10-0e6ef4919f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:40.865218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472e2ee-37b4-4d3f-bbd9-3f2746bf3557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:40.866979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472e2ee-37b4-4d3f-bbd9-3f2746bf3557 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e472e2ee-37b4-4d3f-bbd9-3f2746bf3557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:40.886653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:40.888526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:40.890172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:40.890172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:40.890172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:40.890172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 282/553] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:40.890172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e2ee-37b4-4d3f-bbd9-3f2746bf3557 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:40.972671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 272/554] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:40.974501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588472c6-a7c8-4e47-bd10-0e6ef4919f21 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:41.002706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:41.094569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e2ee-37b4-4d3f-bbd9-3f2746bf3557 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:41.120638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:41.180107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588472c6-a7c8-4e47-bd10-0e6ef4919f21 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:41.195567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac402d-74a9-47a0-acfe-1161ef31833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:41.196230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac402d-74a9-47a0-acfe-1161ef31833d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.196516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac402d-74a9-47a0-acfe-1161ef31833d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.196794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac402d-74a9-47a0-acfe-1161ef31833d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:41.218922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac402d-74a9-47a0-acfe-1161ef31833d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:41.220622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac402d-74a9-47a0-acfe-1161ef31833d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-edac402d-74a9-47a0-acfe-1161ef31833d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:41.248317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:41.250582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:41.252100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 283/555] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:23:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:41.258877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac402d-74a9-47a0-acfe-1161ef31833d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:41.271306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01679a84-fbc5-49b0-b074-3f0e1415a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:41.273379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01679a84-fbc5-49b0-b074-3f0e1415a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.274239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01679a84-fbc5-49b0-b074-3f0e1415a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.274976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01679a84-fbc5-49b0-b074-3f0e1415a1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:41.276906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01679a84-fbc5-49b0-b074-3f0e1415a1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:41.277747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01679a84-fbc5-49b0-b074-3f0e1415a1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:41.280630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01679a84-fbc5-49b0-b074-3f0e1415a1e5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:41.300469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 273/556] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:41.311109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01679a84-fbc5-49b0-b074-3f0e1415a1e5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:41.500851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01679a84-fbc5-49b0-b074-3f0e1415a1e5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:41.544314 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac402d-74a9-47a0-acfe-1161ef31833d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:41.551756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:41.560678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca702b-4f3f-44ed-92cb-89c4f84dec18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:41.561597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca702b-4f3f-44ed-92cb-89c4f84dec18 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.561980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca702b-4f3f-44ed-92cb-89c4f84dec18 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.562307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca702b-4f3f-44ed-92cb-89c4f84dec18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:41.568225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:41.577172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca702b-4f3f-44ed-92cb-89c4f84dec18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:41.577620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca702b-4f3f-44ed-92cb-89c4f84dec18 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:41.595300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a788673-7ab4-4c1d-9e1a-133f8d14ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:41.595911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a788673-7ab4-4c1d-9e1a-133f8d14ca65 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-afca702b-4f3f-44ed-92cb-89c4f84dec18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:41.596124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:41.599542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:23:41.602385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:41.604742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a788673-7ab4-4c1d-9e1a-133f8d14ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.604742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a788673-7ab4-4c1d-9e1a-133f8d14ca65 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:41.604987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 274/557] 10.222.0.100 () {56 vars in 1098 bytes} [Thu Sep 1 17:23:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:41.605131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a788673-7ab4-4c1d-9e1a-133f8d14ca65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:41.605131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a788673-7ab4-4c1d-9e1a-133f8d14ca65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:41.605383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca702b-4f3f-44ed-92cb-89c4f84dec18 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:41.605510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a788673-7ab4-4c1d-9e1a-133f8d14ca65 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:41.635484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 284/558] 10.222.0.100 () {56 vars in 1140 bytes} [Thu Sep 1 17:23:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:41.638008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f625-3cbe-48b7-9314-02241116da47 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.638299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f625-3cbe-48b7-9314-02241116da47 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.638567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f625-3cbe-48b7-9314-02241116da47 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:41.645212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 285/559] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:41.645212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21fff9-0b97-4fcc-bdd9-6358c763fb0d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.645212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21fff9-0b97-4fcc-bdd9-6358c763fb0d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.645212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21fff9-0b97-4fcc-bdd9-6358c763fb0d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:41.645212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 286/560] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:41.645212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21fff9-0b97-4fcc-bdd9-6358c763fb0d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:41.780173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca702b-4f3f-44ed-92cb-89c4f84dec18 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:41.792646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21fff9-0b97-4fcc-bdd9-6358c763fb0d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:41.833215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9c4f1-bef1-443e-b79e-39452c6fc0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:41.833869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9c4f1-bef1-443e-b79e-39452c6fc0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.834150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9c4f1-bef1-443e-b79e-39452c6fc0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.834982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9c4f1-bef1-443e-b79e-39452c6fc0b4 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:41.835491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55e263-528d-4aec-8c76-6220b6565b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:41.835741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9c4f1-bef1-443e-b79e-39452c6fc0b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:41.836053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9c4f1-bef1-443e-b79e-39452c6fc0b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:41.836234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55e263-528d-4aec-8c76-6220b6565b29 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.836413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55e263-528d-4aec-8c76-6220b6565b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.836653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55e263-528d-4aec-8c76-6220b6565b29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:41.837512 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9c4f1-bef1-443e-b79e-39452c6fc0b4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:41.851442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55e263-528d-4aec-8c76-6220b6565b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5d55e263-528d-4aec-8c76-6220b6565b29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:41.859203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:41.860913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 275/561] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:41.865152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55e263-528d-4aec-8c76-6220b6565b29 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:41.870884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 287/562] 10.222.0.100 () {56 vars in 1138 bytes} [Thu Sep 1 17:23:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:41.872959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378badd-0389-4324-b741-486ae387bbb0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.873384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378badd-0389-4324-b741-486ae387bbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.873710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378badd-0389-4324-b741-486ae387bbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:41.897656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6378badd-0389-4324-b741-486ae387bbb0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:41.899717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6378badd-0389-4324-b741-486ae387bbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:41.933865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:41.945883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:41.985440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 288/563] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:41.987338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38176e10-1bb5-486a-88f9-21c37b52a96c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:41.988091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38176e10-1bb5-486a-88f9-21c37b52a96c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:41.988573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38176e10-1bb5-486a-88f9-21c37b52a96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:42.020337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-38176e10-1bb5-486a-88f9-21c37b52a96c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:42.020606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-38176e10-1bb5-486a-88f9-21c37b52a96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:42.020766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55e263-528d-4aec-8c76-6220b6565b29 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:42.076096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3890008-7e0e-4cea-8258-173d598f020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:42.076739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3890008-7e0e-4cea-8258-173d598f020c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:42.077082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3890008-7e0e-4cea-8258-173d598f020c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:42.077429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3890008-7e0e-4cea-8258-173d598f020c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:42.133641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 289/564] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:42.133641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38176e10-1bb5-486a-88f9-21c37b52a96c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:42.190584 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3890008-7e0e-4cea-8258-173d598f020c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:42.192420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3890008-7e0e-4cea-8258-173d598f020c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:42.269516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:42.297827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:42.314469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38176e10-1bb5-486a-88f9-21c37b52a96c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:42.335993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 276/565] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:41 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:42.336781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3890008-7e0e-4cea-8258-173d598f020c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:42.367490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fa21d-09b4-4ff7-b957-5846cd0dfede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:42.368110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fa21d-09b4-4ff7-b957-5846cd0dfede None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:42.368394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fa21d-09b4-4ff7-b957-5846cd0dfede None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:42.368795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fa21d-09b4-4ff7-b957-5846cd0dfede None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:42.369489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fa21d-09b4-4ff7-b957-5846cd0dfede None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:42.369894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fa21d-09b4-4ff7-b957-5846cd0dfede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:42.371294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fa21d-09b4-4ff7-b957-5846cd0dfede None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:42.386280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 290/566] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:42 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:42.387269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fa21d-09b4-4ff7-b957-5846cd0dfede None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:42.497952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3890008-7e0e-4cea-8258-173d598f020c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:42.537201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fa21d-09b4-4ff7-b957-5846cd0dfede None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:42.545671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220581e-7be5-494b-8701-2ebeefbc074a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:42.546315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220581e-7be5-494b-8701-2ebeefbc074a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:42.546595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220581e-7be5-494b-8701-2ebeefbc074a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:42.546861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220581e-7be5-494b-8701-2ebeefbc074a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:42.564146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220581e-7be5-494b-8701-2ebeefbc074a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:42.565742 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220581e-7be5-494b-8701-2ebeefbc074a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:42.576679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4b60e-31c7-4f07-acd5-7a24f02caf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:42.577464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4b60e-31c7-4f07-acd5-7a24f02caf8f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:42.577672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4b60e-31c7-4f07-acd5-7a24f02caf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:42.577861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4b60e-31c7-4f07-acd5-7a24f02caf8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1220581e-7be5-494b-8701-2ebeefbc074a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:42.582152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:42.583432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:42.584379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:42.584379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:42.584379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:42.584379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 277/567] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:42.591566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220581e-7be5-494b-8701-2ebeefbc074a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:42.594420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:42.596054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4b60e-31c7-4f07-acd5-7a24f02caf8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-37e4b60e-31c7-4f07-acd5-7a24f02caf8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:42.607263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:42.608374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:42.609212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:42.609873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 291/568] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:42.610929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4b60e-31c7-4f07-acd5-7a24f02caf8f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:42.632717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:42.722604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220581e-7be5-494b-8701-2ebeefbc074a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:42.745801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4b60e-31c7-4f07-acd5-7a24f02caf8f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:42.760056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d942e-4cb0-442a-b687-715ef86f69bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:42.760774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d942e-4cb0-442a-b687-715ef86f69bc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:42.761153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d942e-4cb0-442a-b687-715ef86f69bc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:42.761434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d942e-4cb0-442a-b687-715ef86f69bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:42.774393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d942e-4cb0-442a-b687-715ef86f69bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:42.775889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d942e-4cb0-442a-b687-715ef86f69bc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:42.791829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945de8dc-72d8-4afb-9639-95fbcb8caf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:42.792481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945de8dc-72d8-4afb-9639-95fbcb8caf7f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:42.792767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945de8dc-72d8-4afb-9639-95fbcb8caf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:42.793041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945de8dc-72d8-4afb-9639-95fbcb8caf7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-564d942e-4cb0-442a-b687-715ef86f69bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:42.798678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:42.800718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:42.802630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:42.802630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:42.802630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:42.802630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 278/569] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:42.802630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d942e-4cb0-442a-b687-715ef86f69bc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:42.809241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945de8dc-72d8-4afb-9639-95fbcb8caf7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:42.810736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945de8dc-72d8-4afb-9639-95fbcb8caf7f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-945de8dc-72d8-4afb-9639-95fbcb8caf7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:42.827627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:42.828653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:42.829808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:42.829808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:42.829808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:42.829808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 292/570] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:42.829808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945de8dc-72d8-4afb-9639-95fbcb8caf7f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:42.870716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:42.913158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:42.925769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d942e-4cb0-442a-b687-715ef86f69bc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:42.954413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945de8dc-72d8-4afb-9639-95fbcb8caf7f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:42.964184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9ee6b-40a0-42e9-b108-784fe327369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:42.964966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9ee6b-40a0-42e9-b108-784fe327369e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:42.965230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9ee6b-40a0-42e9-b108-784fe327369e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:42.965505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9ee6b-40a0-42e9-b108-784fe327369e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:42.977709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9ee6b-40a0-42e9-b108-784fe327369e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1fb9ee6b-40a0-42e9-b108-784fe327369e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:42.983156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:42.984373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 279/571] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:42.985224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9ee6b-40a0-42e9-b108-784fe327369e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:42.997464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba0ced-2262-4f34-86c9-62ec9548baf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:42.998114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba0ced-2262-4f34-86c9-62ec9548baf2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:42.998402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba0ced-2262-4f34-86c9-62ec9548baf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:42.998673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba0ced-2262-4f34-86c9-62ec9548baf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:42.999333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba0ced-2262-4f34-86c9-62ec9548baf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:42.999638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba0ced-2262-4f34-86c9-62ec9548baf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.001042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba0ced-2262-4f34-86c9-62ec9548baf2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.014182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 293/572] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:43.015072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba0ced-2262-4f34-86c9-62ec9548baf2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:43.119950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9ee6b-40a0-42e9-b108-784fe327369e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.133158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba0ced-2262-4f34-86c9-62ec9548baf2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.137531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:43.163841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c488d2f-b2d1-4256-9a3d-bb5285672c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.164606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c488d2f-b2d1-4256-9a3d-bb5285672c59 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.164755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c488d2f-b2d1-4256-9a3d-bb5285672c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.165042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c488d2f-b2d1-4256-9a3d-bb5285672c59 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.165213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:43.165760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c488d2f-b2d1-4256-9a3d-bb5285672c59 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:43.166065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c488d2f-b2d1-4256-9a3d-bb5285672c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.167692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c488d2f-b2d1-4256-9a3d-bb5285672c59 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.176831 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c7d1a-2be7-43a8-b657-75f9f5f341b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.177325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 280/573] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:42 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:43.177512 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c7d1a-2be7-43a8-b657-75f9f5f341b7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.177782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c7d1a-2be7-43a8-b657-75f9f5f341b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.178825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c7d1a-2be7-43a8-b657-75f9f5f341b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.179433 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c488d2f-b2d1-4256-9a3d-bb5285672c59 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:43.180032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c7d1a-2be7-43a8-b657-75f9f5f341b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:43.180345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c7d1a-2be7-43a8-b657-75f9f5f341b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.182053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c7d1a-2be7-43a8-b657-75f9f5f341b7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.194769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 294/574] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:43.195645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c7d1a-2be7-43a8-b657-75f9f5f341b7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:43.307497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c488d2f-b2d1-4256-9a3d-bb5285672c59 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.339236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c7d1a-2be7-43a8-b657-75f9f5f341b7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.363118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4e5dd-2e5f-447e-8ec8-8d608864d143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.363731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4e5dd-2e5f-447e-8ec8-8d608864d143 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.364008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4e5dd-2e5f-447e-8ec8-8d608864d143 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.364279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4e5dd-2e5f-447e-8ec8-8d608864d143 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.364925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4e5dd-2e5f-447e-8ec8-8d608864d143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:43.365229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4e5dd-2e5f-447e-8ec8-8d608864d143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.367120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4e5dd-2e5f-447e-8ec8-8d608864d143 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.380258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd7db0-1f1d-4bd2-9cb0-1f15e26b8cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.380930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd7db0-1f1d-4bd2-9cb0-1f15e26b8cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.381629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd7db0-1f1d-4bd2-9cb0-1f15e26b8cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.381925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd7db0-1f1d-4bd2-9cb0-1f15e26b8cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.382731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd7db0-1f1d-4bd2-9cb0-1f15e26b8cc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:43.383036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd7db0-1f1d-4bd2-9cb0-1f15e26b8cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.384494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd7db0-1f1d-4bd2-9cb0-1f15e26b8cc9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.385490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 281/575] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:43.386375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4e5dd-2e5f-447e-8ec8-8d608864d143 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:43.399801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 295/576] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:43.400999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd7db0-1f1d-4bd2-9cb0-1f15e26b8cc9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:43.432326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:43.458470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:43.513795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4e5dd-2e5f-447e-8ec8-8d608864d143 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.517272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd7db0-1f1d-4bd2-9cb0-1f15e26b8cc9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.554462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6610b1-1fce-426c-bc58-90fac5e1e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.555109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6610b1-1fce-426c-bc58-90fac5e1e88e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.555385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6610b1-1fce-426c-bc58-90fac5e1e88e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.555659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6610b1-1fce-426c-bc58-90fac5e1e88e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.560594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68d8d8-61a4-4c60-b065-b8599d908b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.561228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68d8d8-61a4-4c60-b065-b8599d908b02 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.561513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68d8d8-61a4-4c60-b065-b8599d908b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.561814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68d8d8-61a4-4c60-b065-b8599d908b02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.563490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6610b1-1fce-426c-bc58-90fac5e1e88e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.565056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6610b1-1fce-426c-bc58-90fac5e1e88e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.573872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68d8d8-61a4-4c60-b065-b8599d908b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.575376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68d8d8-61a4-4c60-b065-b8599d908b02 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2e6610b1-1fce-426c-bc58-90fac5e1e88e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:43.577756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:43.579219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:43.581092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:43.581092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:43.581092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:43.581092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:43.581092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:43.581092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:43.581092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:43.581092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:43.583775 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 282/577] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:43.584519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6610b1-1fce-426c-bc58-90fac5e1e88e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6c68d8d8-61a4-4c60-b065-b8599d908b02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:43.591283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:43.592645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:43.593548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:43.593548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:43.593548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:43.593548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 296/578] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:43.593548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68d8d8-61a4-4c60-b065-b8599d908b02 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:43.704886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:43.712760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6610b1-1fce-426c-bc58-90fac5e1e88e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.731441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:43.734962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68d8d8-61a4-4c60-b065-b8599d908b02 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.752408 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef244c-8b32-48e1-b522-0bf36cf4ac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.753049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef244c-8b32-48e1-b522-0bf36cf4ac1b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.753324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef244c-8b32-48e1-b522-0bf36cf4ac1b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.753615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef244c-8b32-48e1-b522-0bf36cf4ac1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.763617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef244c-8b32-48e1-b522-0bf36cf4ac1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.765076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef244c-8b32-48e1-b522-0bf36cf4ac1b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.773398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181d2a9-5284-4b95-a5f7-0e157da5bfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.774065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181d2a9-5284-4b95-a5f7-0e157da5bfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.774346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181d2a9-5284-4b95-a5f7-0e157da5bfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.774618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181d2a9-5284-4b95-a5f7-0e157da5bfe2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a1ef244c-8b32-48e1-b522-0bf36cf4ac1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:43.777483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:43.779209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 283/579] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:43.780260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef244c-8b32-48e1-b522-0bf36cf4ac1b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:43.789096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181d2a9-5284-4b95-a5f7-0e157da5bfe2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.790609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181d2a9-5284-4b95-a5f7-0e157da5bfe2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f181d2a9-5284-4b95-a5f7-0e157da5bfe2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:43.803747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:43.806799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 297/580] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:43.809014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181d2a9-5284-4b95-a5f7-0e157da5bfe2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:43.903946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef244c-8b32-48e1-b522-0bf36cf4ac1b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.930099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181d2a9-5284-4b95-a5f7-0e157da5bfe2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:43.951646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960b387-3280-4d8e-87b6-579cea99fdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.952287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960b387-3280-4d8e-87b6-579cea99fdda None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.952565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960b387-3280-4d8e-87b6-579cea99fdda None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.952834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960b387-3280-4d8e-87b6-579cea99fdda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.969054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960b387-3280-4d8e-87b6-579cea99fdda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.971643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960b387-3280-4d8e-87b6-579cea99fdda None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.979653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fb45e-48d4-493a-ab98-51becc16f8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:43.980602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fb45e-48d4-493a-ab98-51becc16f8ee None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:43.981054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fb45e-48d4-493a-ab98-51becc16f8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:43.981505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fb45e-48d4-493a-ab98-51becc16f8ee None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:43.982702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fb45e-48d4-493a-ab98-51becc16f8ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:43.983193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fb45e-48d4-493a-ab98-51becc16f8ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:43.985386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fb45e-48d4-493a-ab98-51becc16f8ee None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2960b387-3280-4d8e-87b6-579cea99fdda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:43.997998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:43.999579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:44.001171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:44.001171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:44.001171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:44.001171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 284/581] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:44.001171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960b387-3280-4d8e-87b6-579cea99fdda None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.011069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 298/582] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:44.011961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fb45e-48d4-493a-ab98-51becc16f8ee None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.015817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:44.047517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:44.133285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960b387-3280-4d8e-87b6-579cea99fdda None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:44.143555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fb45e-48d4-493a-ab98-51becc16f8ee None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:44.175322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ec4c7-fd9e-4272-aa2e-b38b17efd92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:44.175930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ec4c7-fd9e-4272-aa2e-b38b17efd92c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.176207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ec4c7-fd9e-4272-aa2e-b38b17efd92c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.176475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ec4c7-fd9e-4272-aa2e-b38b17efd92c None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.177120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ec4c7-fd9e-4272-aa2e-b38b17efd92c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:44.177416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ec4c7-fd9e-4272-aa2e-b38b17efd92c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:44.178897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ec4c7-fd9e-4272-aa2e-b38b17efd92c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:44.193312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 285/583] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:44.194209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ec4c7-fd9e-4272-aa2e-b38b17efd92c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.204168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633a7d6-6fdd-4c74-89c8-a41a1117db33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:44.204774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633a7d6-6fdd-4c74-89c8-a41a1117db33 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.205058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633a7d6-6fdd-4c74-89c8-a41a1117db33 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.205345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633a7d6-6fdd-4c74-89c8-a41a1117db33 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.206181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633a7d6-6fdd-4c74-89c8-a41a1117db33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:44.206494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633a7d6-6fdd-4c74-89c8-a41a1117db33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:44.208319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633a7d6-6fdd-4c74-89c8-a41a1117db33 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:44.221938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 299/584] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:44.225793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633a7d6-6fdd-4c74-89c8-a41a1117db33 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.278326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:44.312839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ec4c7-fd9e-4272-aa2e-b38b17efd92c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:44.324641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:44.352703 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633a7d6-6fdd-4c74-89c8-a41a1117db33 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:44.367176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b36ab8-c18f-4abd-8934-c8e1c34447f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:44.368205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b36ab8-c18f-4abd-8934-c8e1c34447f7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.368680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b36ab8-c18f-4abd-8934-c8e1c34447f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.372650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b36ab8-c18f-4abd-8934-c8e1c34447f7 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.373492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b36ab8-c18f-4abd-8934-c8e1c34447f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:44.373823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b36ab8-c18f-4abd-8934-c8e1c34447f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:44.375292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b36ab8-c18f-4abd-8934-c8e1c34447f7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:44.394065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 286/585] 10.222.0.100 () {56 vars in 1142 bytes} [Thu Sep 1 17:23:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:44.394894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b36ab8-c18f-4abd-8934-c8e1c34447f7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.405189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86f0b7-fb3a-4e87-819b-2ea676a827fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:44.405843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86f0b7-fb3a-4e87-819b-2ea676a827fd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.406252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86f0b7-fb3a-4e87-819b-2ea676a827fd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.406567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86f0b7-fb3a-4e87-819b-2ea676a827fd None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.407228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86f0b7-fb3a-4e87-819b-2ea676a827fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:44.407532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86f0b7-fb3a-4e87-819b-2ea676a827fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:44.409216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86f0b7-fb3a-4e87-819b-2ea676a827fd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:44.425101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 300/586] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:44.425823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86f0b7-fb3a-4e87-819b-2ea676a827fd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.524789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b36ab8-c18f-4abd-8934-c8e1c34447f7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:44.542928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86f0b7-fb3a-4e87-819b-2ea676a827fd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:44.558138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:44.569449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e37966-0d8d-4599-ac69-8a3fba67a901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:44.570156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e37966-0d8d-4599-ac69-8a3fba67a901 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.570428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e37966-0d8d-4599-ac69-8a3fba67a901 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.570695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e37966-0d8d-4599-ac69-8a3fba67a901 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/c34f6f3e93b54a5d92b6f2f9f0de40f6/roles/3e624c5fdefb4c03bd915bbc9d097212` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.587228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e37966-0d8d-4599-ac69-8a3fba67a901 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=c34f6f3e93b54a5d92b6f2f9f0de40f6, role_id=3e624c5fdefb4c03bd915bbc9d097212)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:44.587447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152f715-c060-42fd-a1c5-2a695cf4dd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:44.587998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f715-c060-42fd-a1c5-2a695cf4dd74 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.588279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f715-c060-42fd-a1c5-2a695cf4dd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.588551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f715-c060-42fd-a1c5-2a695cf4dd74 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.588832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e37966-0d8d-4599-ac69-8a3fba67a901 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:44.597304 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152f715-c060-42fd-a1c5-2a695cf4dd74 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:44.598920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152f715-c060-42fd-a1c5-2a695cf4dd74 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:44.602610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:44.604110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 287/587] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:23:44 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/c34f6f3e93b54a5d92b6f2f9f0de40f6/roles/3e624c5fdefb4c03bd915bbc9d097212 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:44.605025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e37966-0d8d-4599-ac69-8a3fba67a901 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0152f715-c060-42fd-a1c5-2a695cf4dd74 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:44.620723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:44.622885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:23:44.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:44.627551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 301/588] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:44.628242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152f715-c060-42fd-a1c5-2a695cf4dd74 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.795270 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e37966-0d8d-4599-ac69-8a3fba67a901 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:44.795484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152f715-c060-42fd-a1c5-2a695cf4dd74 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:44.836499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c74c6-6b44-4bfa-a17b-e18b2e26c405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:44.837121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c74c6-6b44-4bfa-a17b-e18b2e26c405 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.837404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c74c6-6b44-4bfa-a17b-e18b2e26c405 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.837716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c74c6-6b44-4bfa-a17b-e18b2e26c405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.838478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c74c6-6b44-4bfa-a17b-e18b2e26c405 None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=f3fa2499f5594b7798f6fe4443a14f4c) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:44.838782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c74c6-6b44-4bfa-a17b-e18b2e26c405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:44.840272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c74c6-6b44-4bfa-a17b-e18b2e26c405 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:44.846124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3453e6-8889-4ef6-81a4-60f773055eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:44.846996 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3453e6-8889-4ef6-81a4-60f773055eec None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:44.847420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3453e6-8889-4ef6-81a4-60f773055eec None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:44.847755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 288/589] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:44 2022] GET /identity/v3/role_assignments?user.id=f3fa2499f5594b7798f6fe4443a14f4c&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:44.848009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3453e6-8889-4ef6-81a4-60f773055eec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:44.848602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c74c6-6b44-4bfa-a17b-e18b2e26c405 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.866850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3453e6-8889-4ef6-81a4-60f773055eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:44.868415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3453e6-8889-4ef6-81a4-60f773055eec None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3f3453e6-8889-4ef6-81a4-60f773055eec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:44.886507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:44.887714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:44.888813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:44.888813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:44.888813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:44.888813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 302/590] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:44.888813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3453e6-8889-4ef6-81a4-60f773055eec None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:44.910204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:44.968704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:44.997920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c74c6-6b44-4bfa-a17b-e18b2e26c405 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:45.040344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3453e6-8889-4ef6-81a4-60f773055eec None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:45.059836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e425991-318e-490a-8c88-3a7718a7067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:45.060383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e425991-318e-490a-8c88-3a7718a7067d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:45.060664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e425991-318e-490a-8c88-3a7718a7067d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:45.060983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e425991-318e-490a-8c88-3a7718a7067d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:45.071097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e425991-318e-490a-8c88-3a7718a7067d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:45.072637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e425991-318e-490a-8c88-3a7718a7067d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8e425991-318e-490a-8c88-3a7718a7067d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:45.085365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:45.087200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 289/591] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:45.089003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e425991-318e-490a-8c88-3a7718a7067d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:45.122878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36846a-4a90-4bc7-b8ff-42c8376a44bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:45.124683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36846a-4a90-4bc7-b8ff-42c8376a44bf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:45.126004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36846a-4a90-4bc7-b8ff-42c8376a44bf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:45.126846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36846a-4a90-4bc7-b8ff-42c8376a44bf None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:45.128541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36846a-4a90-4bc7-b8ff-42c8376a44bf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:45.129364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36846a-4a90-4bc7-b8ff-42c8376a44bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:45.132756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36846a-4a90-4bc7-b8ff-42c8376a44bf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:45.169566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 303/592] 10.222.0.100 () {56 vars in 1142 bytes} [Thu Sep 1 17:23:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:45.171619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36846a-4a90-4bc7-b8ff-42c8376a44bf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:45.253215 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:45.257515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e425991-318e-490a-8c88-3a7718a7067d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:45.301937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b001f0-d240-43f1-8aa0-d78750f6191e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:45.302709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b001f0-d240-43f1-8aa0-d78750f6191e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:45.303146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b001f0-d240-43f1-8aa0-d78750f6191e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:45.303570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b001f0-d240-43f1-8aa0-d78750f6191e None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:45.304367 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b001f0-d240-43f1-8aa0-d78750f6191e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:45.304767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b001f0-d240-43f1-8aa0-d78750f6191e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:45.306333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b001f0-d240-43f1-8aa0-d78750f6191e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:45.330571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 290/593] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:45.332881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b001f0-d240-43f1-8aa0-d78750f6191e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:45.351923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36846a-4a90-4bc7-b8ff-42c8376a44bf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:45.370325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:45.391485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a469d4-2971-421d-bb67-bd42bf3a6091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:45.392105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a469d4-2971-421d-bb67-bd42bf3a6091 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:45.392387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a469d4-2971-421d-bb67-bd42bf3a6091 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:45.392666 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a469d4-2971-421d-bb67-bd42bf3a6091 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:45.393409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a469d4-2971-421d-bb67-bd42bf3a6091 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:45.393768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a469d4-2971-421d-bb67-bd42bf3a6091 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:45.395754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a469d4-2971-421d-bb67-bd42bf3a6091 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:45.423721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 304/594] 10.222.0.100 () {56 vars in 1144 bytes} [Thu Sep 1 17:23:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:45.424659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a469d4-2971-421d-bb67-bd42bf3a6091 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:45.467213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b001f0-d240-43f1-8aa0-d78750f6191e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:45.508028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d543f1f-ed6f-4042-a6dc-06dc5ae2e2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:45.508665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d543f1f-ed6f-4042-a6dc-06dc5ae2e2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:45.509009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d543f1f-ed6f-4042-a6dc-06dc5ae2e2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:45.509335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d543f1f-ed6f-4042-a6dc-06dc5ae2e2d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:45.522331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d543f1f-ed6f-4042-a6dc-06dc5ae2e2d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:45.523900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d543f1f-ed6f-4042-a6dc-06dc5ae2e2d8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9d543f1f-ed6f-4042-a6dc-06dc5ae2e2d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:45.540896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:45.542142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:45.543133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:45.543133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:45.543133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:45.543133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 291/595] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:45.543360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d543f1f-ed6f-4042-a6dc-06dc5ae2e2d8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:45.560815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:45.562776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a469d4-2971-421d-bb67-bd42bf3a6091 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:45.610157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad93dc-867d-47fc-a89c-c31da113bbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:45.612163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad93dc-867d-47fc-a89c-c31da113bbee None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:45.612610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad93dc-867d-47fc-a89c-c31da113bbee None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:45.613033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad93dc-867d-47fc-a89c-c31da113bbee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:45.623691 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad93dc-867d-47fc-a89c-c31da113bbee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:45.626435 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad93dc-867d-47fc-a89c-c31da113bbee None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-33ad93dc-867d-47fc-a89c-c31da113bbee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:45.649631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:45.651616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:45.653818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:45.653818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:45.653818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:45.653818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:45.653818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:45.653818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:45.653818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:45.653818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:45.669289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 305/596] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:45.670431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad93dc-867d-47fc-a89c-c31da113bbee None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:45.706899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:45.708356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d543f1f-ed6f-4042-a6dc-06dc5ae2e2d8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:45.770621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da88547-781d-4842-97d2-91ef34d3451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:45.771852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da88547-781d-4842-97d2-91ef34d3451d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:45.772429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da88547-781d-4842-97d2-91ef34d3451d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:45.772981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da88547-781d-4842-97d2-91ef34d3451d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:45.796494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da88547-781d-4842-97d2-91ef34d3451d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:45.799511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da88547-781d-4842-97d2-91ef34d3451d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:45.814195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad93dc-867d-47fc-a89c-c31da113bbee None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5da88547-781d-4842-97d2-91ef34d3451d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:45.828146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:45.830360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:45.832454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:45.832454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:45.832454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:45.836450 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 292/597] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:45.838418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da88547-781d-4842-97d2-91ef34d3451d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:45.858123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4816062-59ca-456c-8b12-b7991af8c260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:45.858846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816062-59ca-456c-8b12-b7991af8c260 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:45.859169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816062-59ca-456c-8b12-b7991af8c260 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:45.859505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816062-59ca-456c-8b12-b7991af8c260 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:45.860290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4816062-59ca-456c-8b12-b7991af8c260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:45.860585 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4816062-59ca-456c-8b12-b7991af8c260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:45.862033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4816062-59ca-456c-8b12-b7991af8c260 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:45.881434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 306/598] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:45.881434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4816062-59ca-456c-8b12-b7991af8c260 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:45.962643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:45.981027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:46.002789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4816062-59ca-456c-8b12-b7991af8c260 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:46.028104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da88547-781d-4842-97d2-91ef34d3451d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:46.049510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75214a5a-a547-42d0-8212-772218082c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:46.050559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75214a5a-a547-42d0-8212-772218082c55 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.051050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75214a5a-a547-42d0-8212-772218082c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.051531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75214a5a-a547-42d0-8212-772218082c55 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.052439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75214a5a-a547-42d0-8212-772218082c55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:46.053002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75214a5a-a547-42d0-8212-772218082c55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:46.054987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75214a5a-a547-42d0-8212-772218082c55 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:46.069294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56666c-b80b-4f0c-8b75-f02cc5b8f692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:46.069929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 307/599] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:46.070787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56666c-b80b-4f0c-8b75-f02cc5b8f692 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.071244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56666c-b80b-4f0c-8b75-f02cc5b8f692 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.071523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56666c-b80b-4f0c-8b75-f02cc5b8f692 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.071775 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75214a5a-a547-42d0-8212-772218082c55 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:46.072325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56666c-b80b-4f0c-8b75-f02cc5b8f692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:46.072824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56666c-b80b-4f0c-8b75-f02cc5b8f692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:46.075910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56666c-b80b-4f0c-8b75-f02cc5b8f692 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:46.092549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 293/600] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:46.093449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56666c-b80b-4f0c-8b75-f02cc5b8f692 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:46.191120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75214a5a-a547-42d0-8212-772218082c55 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:46.218126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56666c-b80b-4f0c-8b75-f02cc5b8f692 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:46.233143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d487c73-9051-4976-9db5-692f05490a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:46.234326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d487c73-9051-4976-9db5-692f05490a8d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.234920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d487c73-9051-4976-9db5-692f05490a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.235398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d487c73-9051-4976-9db5-692f05490a8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.244456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d487c73-9051-4976-9db5-692f05490a8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:46.246488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:46.247739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d487c73-9051-4976-9db5-692f05490a8d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:46.259681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d845c64-e21b-43c5-bd14-7b5a50488547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:46.260333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d845c64-e21b-43c5-bd14-7b5a50488547 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.260615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d845c64-e21b-43c5-bd14-7b5a50488547 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.260939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d845c64-e21b-43c5-bd14-7b5a50488547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0d487c73-9051-4976-9db5-692f05490a8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:46.265398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:46.266355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:46.267267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:46.267267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:46.267267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:46.267267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:46.267267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:46.267267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:46.267267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:46.267267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:46.269123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d845c64-e21b-43c5-bd14-7b5a50488547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:46.273868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d845c64-e21b-43c5-bd14-7b5a50488547 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:46.274023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 308/601] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:46.275198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d487c73-9051-4976-9db5-692f05490a8d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:46.290936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2d845c64-e21b-43c5-bd14-7b5a50488547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:46.292096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:46.293093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:46.294207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:46.294207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:46.294207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:46.294207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:46.294207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:46.294207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:46.294207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:46.294207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:46.296308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 294/602] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:46.297359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d845c64-e21b-43c5-bd14-7b5a50488547 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:46.402491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d487c73-9051-4976-9db5-692f05490a8d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:46.408621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d845c64-e21b-43c5-bd14-7b5a50488547 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:46.442511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6516a0-9fe3-4623-ab23-d3cc541e67ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:46.443291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6516a0-9fe3-4623-ab23-d3cc541e67ab None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.443864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6516a0-9fe3-4623-ab23-d3cc541e67ab None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.444245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6516a0-9fe3-4623-ab23-d3cc541e67ab None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/e0891d81e6e24ad99a9b532059c8c2af/roles/3e624c5fdefb4c03bd915bbc9d097212` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.450776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a6a2e-1579-4ce6-996e-4007f610b43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:46.451683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a6a2e-1579-4ce6-996e-4007f610b43f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.452141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a6a2e-1579-4ce6-996e-4007f610b43f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.452565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a6a2e-1579-4ce6-996e-4007f610b43f None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.453421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a6a2e-1579-4ce6-996e-4007f610b43f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:46.453921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a6a2e-1579-4ce6-996e-4007f610b43f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:46.455555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a6a2e-1579-4ce6-996e-4007f610b43f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:46.463991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6516a0-9fe3-4623-ab23-d3cc541e67ab None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=e0891d81e6e24ad99a9b532059c8c2af, role_id=3e624c5fdefb4c03bd915bbc9d097212)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:46.465815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6516a0-9fe3-4623-ab23-d3cc541e67ab None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:46.470924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 295/603] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:46.472121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a6a2e-1579-4ce6-996e-4007f610b43f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:46.487244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 309/604] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:23:46 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/e0891d81e6e24ad99a9b532059c8c2af/roles/3e624c5fdefb4c03bd915bbc9d097212 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:46.488456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6516a0-9fe3-4623-ab23-d3cc541e67ab None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:46.573614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:46.581615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:46.625185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6516a0-9fe3-4623-ab23-d3cc541e67ab None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:46.671766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da483b8c-9560-4917-9531-16eab57a0f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:46.672467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483b8c-9560-4917-9531-16eab57a0f52 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.672750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483b8c-9560-4917-9531-16eab57a0f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.673028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483b8c-9560-4917-9531-16eab57a0f52 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/1d47d8e089f446d3bb4b6609d7256ac2/roles/3e624c5fdefb4c03bd915bbc9d097212` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.693825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483b8c-9560-4917-9531-16eab57a0f52 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=1d47d8e089f446d3bb4b6609d7256ac2, role_id=3e624c5fdefb4c03bd915bbc9d097212)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:46.696168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483b8c-9560-4917-9531-16eab57a0f52 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:46.717885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 310/605] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:23:46 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/1d47d8e089f446d3bb4b6609d7256ac2/roles/3e624c5fdefb4c03bd915bbc9d097212 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:46.723133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053976fd-7cef-4a35-bb11-7f01b168ea3b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.723499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053976fd-7cef-4a35-bb11-7f01b168ea3b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.729603 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053976fd-7cef-4a35-bb11-7f01b168ea3b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.729603 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 311/606] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:46 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:46.734243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e0782-a95e-456f-adcb-40ed536a51bc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.734589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e0782-a95e-456f-adcb-40ed536a51bc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.735804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e0782-a95e-456f-adcb-40ed536a51bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.750224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a6a2e-1579-4ce6-996e-4007f610b43f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:46.768235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e17e0782-a95e-456f-adcb-40ed536a51bc None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:46.769542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e17e0782-a95e-456f-adcb-40ed536a51bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:46.802425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf7a45-6b1d-4541-bc06-0a78effc3f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:46.803726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf7a45-6b1d-4541-bc06-0a78effc3f90 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:46.804366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf7a45-6b1d-4541-bc06-0a78effc3f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:46.804874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf7a45-6b1d-4541-bc06-0a78effc3f90 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:46.805993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf7a45-6b1d-4541-bc06-0a78effc3f90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:46.806555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf7a45-6b1d-4541-bc06-0a78effc3f90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:46.808618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf7a45-6b1d-4541-bc06-0a78effc3f90 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:46.823866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 296/607] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:46.830366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf7a45-6b1d-4541-bc06-0a78effc3f90 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:46.917838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 312/608] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:46.999484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf7a45-6b1d-4541-bc06-0a78effc3f90 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:47.001958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:47.039711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b738f4-f02f-4c89-b59c-d46d53e7d521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:47.040456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b738f4-f02f-4c89-b59c-d46d53e7d521 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:47.040744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b738f4-f02f-4c89-b59c-d46d53e7d521 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:47.041024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b738f4-f02f-4c89-b59c-d46d53e7d521 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/bbe180b45f2e49e2abbf6e833ce0d44c/roles/3e624c5fdefb4c03bd915bbc9d097212` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:47.058242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b738f4-f02f-4c89-b59c-d46d53e7d521 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=bbe180b45f2e49e2abbf6e833ce0d44c, role_id=3e624c5fdefb4c03bd915bbc9d097212)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:47.059710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b738f4-f02f-4c89-b59c-d46d53e7d521 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:47.079094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 297/609] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:23:46 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/bbe180b45f2e49e2abbf6e833ce0d44c/roles/3e624c5fdefb4c03bd915bbc9d097212 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:47.277631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e0782-a95e-456f-adcb-40ed536a51bc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:47.336028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:47.410221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9a907-7345-4368-b3ec-645adb4b275e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:47.410511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9a907-7345-4368-b3ec-645adb4b275e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:47.410793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9a907-7345-4368-b3ec-645adb4b275e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:47.412317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 298/610] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:47.423574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa530b42-4187-4eb7-990c-b0765f87ba13 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:47.424370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa530b42-4187-4eb7-990c-b0765f87ba13 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:47.425044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa530b42-4187-4eb7-990c-b0765f87ba13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:47.440536 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e0782-a95e-456f-adcb-40ed536a51bc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:47.464212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fa530b42-4187-4eb7-990c-b0765f87ba13 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:47.467451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fa530b42-4187-4eb7-990c-b0765f87ba13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:47.484285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3386ed-d4f8-4065-809d-9f6d4d715ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:47.485118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3386ed-d4f8-4065-809d-9f6d4d715ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:47.485608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3386ed-d4f8-4065-809d-9f6d4d715ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:47.485986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3386ed-d4f8-4065-809d-9f6d4d715ff3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:47.505131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3386ed-d4f8-4065-809d-9f6d4d715ff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0b3386ed-d4f8-4065-809d-9f6d4d715ff3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:47.515418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:47.516604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:47.517652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:47.520339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 313/611] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:47.528143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3386ed-d4f8-4065-809d-9f6d4d715ff3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:47.588664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 299/612] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:23:47.702006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:47.710211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3386ed-d4f8-4065-809d-9f6d4d715ff3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:47.767578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528ac65-05fd-438b-a040-949376550ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:47.768479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528ac65-05fd-438b-a040-949376550ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:47.768908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528ac65-05fd-438b-a040-949376550ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:47.769337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528ac65-05fd-438b-a040-949376550ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:47.770186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528ac65-05fd-438b-a040-949376550ea1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:47.770633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528ac65-05fd-438b-a040-949376550ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:47.772247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528ac65-05fd-438b-a040-949376550ea1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:47.785040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 314/613] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:47 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:47.791334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa530b42-4187-4eb7-990c-b0765f87ba13 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:47.893432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:47.910333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528ac65-05fd-438b-a040-949376550ea1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:47.917798 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa530b42-4187-4eb7-990c-b0765f87ba13 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:47.968139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e0cf4-2b8d-4d66-8252-10e74a45416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:47.968828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e0cf4-2b8d-4d66-8252-10e74a45416c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:47.969112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e0cf4-2b8d-4d66-8252-10e74a45416c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:47.969386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e0cf4-2b8d-4d66-8252-10e74a45416c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:47.990334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e0cf4-2b8d-4d66-8252-10e74a45416c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:47.992032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e0cf4-2b8d-4d66-8252-10e74a45416c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-296e0cf4-2b8d-4d66-8252-10e74a45416c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:48.019710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:48.020748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:48.022622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:48.022622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:48.022622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:48.022622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 300/614] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:48.029110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e0cf4-2b8d-4d66-8252-10e74a45416c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:48.049034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528ac65-05fd-438b-a040-949376550ea1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:48.088438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89181b-f751-4b54-ab2b-633959afb378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:48.089073 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89181b-f751-4b54-ab2b-633959afb378 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:48.089356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89181b-f751-4b54-ab2b-633959afb378 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:48.089647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89181b-f751-4b54-ab2b-633959afb378 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:48.103975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89181b-f751-4b54-ab2b-633959afb378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-ee89181b-f751-4b54-ab2b-633959afb378 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:48.110088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:48.111054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:48.111889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 315/615] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:48.117780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89181b-f751-4b54-ab2b-633959afb378 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:48.142000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:48.272496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e0cf4-2b8d-4d66-8252-10e74a45416c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:48.275764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89181b-f751-4b54-ab2b-633959afb378 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:48.335849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc04a-a1bc-4cf8-84b4-e686fec3ef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:48.336359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc04a-a1bc-4cf8-84b4-e686fec3ef87 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:48.336644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc04a-a1bc-4cf8-84b4-e686fec3ef87 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:48.336917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc04a-a1bc-4cf8-84b4-e686fec3ef87 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:48.337566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbc04a-a1bc-4cf8-84b4-e686fec3ef87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:48.337890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbc04a-a1bc-4cf8-84b4-e686fec3ef87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:48.339264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbc04a-a1bc-4cf8-84b4-e686fec3ef87 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:48.342273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709b3d1-8420-4957-92c5-17e25ec3755a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:48.342886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709b3d1-8420-4957-92c5-17e25ec3755a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:48.343163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709b3d1-8420-4957-92c5-17e25ec3755a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:48.343435 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709b3d1-8420-4957-92c5-17e25ec3755a None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:48.344101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709b3d1-8420-4957-92c5-17e25ec3755a None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:48.344398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709b3d1-8420-4957-92c5-17e25ec3755a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:48.345777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709b3d1-8420-4957-92c5-17e25ec3755a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:48.349070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:48.363447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 316/616] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:48 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:48.365719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 301/617] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:48.374762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709b3d1-8420-4957-92c5-17e25ec3755a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:48.378637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc04a-a1bc-4cf8-84b4-e686fec3ef87 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:48.478312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:48.489307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709b3d1-8420-4957-92c5-17e25ec3755a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:48.550621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b0fd7-2ccb-47ee-800a-cc418963cdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:48.551262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b0fd7-2ccb-47ee-800a-cc418963cdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:48.551546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b0fd7-2ccb-47ee-800a-cc418963cdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:48.551818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b0fd7-2ccb-47ee-800a-cc418963cdf8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:48.567624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b0fd7-2ccb-47ee-800a-cc418963cdf8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:48.569740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b0fd7-2ccb-47ee-800a-cc418963cdf8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-504b0fd7-2ccb-47ee-800a-cc418963cdf8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:48.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:48.597450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:48.599480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 317/618] 10.222.0.100 () {56 vars in 1098 bytes} [Thu Sep 1 17:23:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:48.604250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b0fd7-2ccb-47ee-800a-cc418963cdf8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:48.631321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc04a-a1bc-4cf8-84b4-e686fec3ef87 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:48.709043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f83890-ce0e-4853-9ec3-16a16aebb47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:48.709891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f83890-ce0e-4853-9ec3-16a16aebb47d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:48.710177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f83890-ce0e-4853-9ec3-16a16aebb47d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:48.710425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f83890-ce0e-4853-9ec3-16a16aebb47d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:48.735375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f83890-ce0e-4853-9ec3-16a16aebb47d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:48.738115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f83890-ce0e-4853-9ec3-16a16aebb47d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e1f83890-ce0e-4853-9ec3-16a16aebb47d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:48.776516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:48.778923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:48.781064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:48.781064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:48.781064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:48.786564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 302/619] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:48.792395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f83890-ce0e-4853-9ec3-16a16aebb47d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:48.858719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:48.881405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b0fd7-2ccb-47ee-800a-cc418963cdf8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:48.929711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f83890-ce0e-4853-9ec3-16a16aebb47d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:48.971684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c79d3-7a1c-4a29-a61a-9c1c33e22fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:48.972347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c79d3-7a1c-4a29-a61a-9c1c33e22fce None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:48.972635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c79d3-7a1c-4a29-a61a-9c1c33e22fce None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:48.972909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c79d3-7a1c-4a29-a61a-9c1c33e22fce None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:48.973677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c79d3-7a1c-4a29-a61a-9c1c33e22fce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:48.973986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c79d3-7a1c-4a29-a61a-9c1c33e22fce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:48.975830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c79d3-7a1c-4a29-a61a-9c1c33e22fce None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:49.008496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbe93f-ba2d-4c58-8781-e204917c2c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:49.009134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbe93f-ba2d-4c58-8781-e204917c2c2f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.009465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbe93f-ba2d-4c58-8781-e204917c2c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.009769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbe93f-ba2d-4c58-8781-e204917c2c2f None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.010462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbe93f-ba2d-4c58-8781-e204917c2c2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:49.010769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbe93f-ba2d-4c58-8781-e204917c2c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:49.012227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbe93f-ba2d-4c58-8781-e204917c2c2f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:49.017550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:49.018347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 318/620] 10.222.0.100 () {56 vars in 1138 bytes} [Thu Sep 1 17:23:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:49.023634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c79d3-7a1c-4a29-a61a-9c1c33e22fce None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:49.048201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 303/621] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:49.052344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbe93f-ba2d-4c58-8781-e204917c2c2f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:49.212540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c79d3-7a1c-4a29-a61a-9c1c33e22fce None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:49.252800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e18d75-6050-4d8a-b7b1-5cedfd5f2553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:49.253421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e18d75-6050-4d8a-b7b1-5cedfd5f2553 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.253749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e18d75-6050-4d8a-b7b1-5cedfd5f2553 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.254023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e18d75-6050-4d8a-b7b1-5cedfd5f2553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.347128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbe93f-ba2d-4c58-8781-e204917c2c2f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:49.384492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:49.397740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e18d75-6050-4d8a-b7b1-5cedfd5f2553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:49.399615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e18d75-6050-4d8a-b7b1-5cedfd5f2553 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:49.405972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:49.493440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ed929-fa52-4c17-a9af-f5e6af891360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:49.494017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ed929-fa52-4c17-a9af-f5e6af891360 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.494313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ed929-fa52-4c17-a9af-f5e6af891360 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.494587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ed929-fa52-4c17-a9af-f5e6af891360 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.503445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ed929-fa52-4c17-a9af-f5e6af891360 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:49.504906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ed929-fa52-4c17-a9af-f5e6af891360 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-382ed929-fa52-4c17-a9af-f5e6af891360 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:49.518463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:49.520361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 304/622] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:23:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e70a-c55c-4964-8c9d-7ad46b777739 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.523558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e70a-c55c-4964-8c9d-7ad46b777739 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.524990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e70a-c55c-4964-8c9d-7ad46b777739 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.526033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 319/624] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:49 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:23:49.533629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e18d75-6050-4d8a-b7b1-5cedfd5f2553 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:49.534123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 305/624] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:49.534123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f3bf1-3494-4dbf-bd94-ba7b6e9ed064 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.534123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f3bf1-3494-4dbf-bd94-ba7b6e9ed064 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.534123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f3bf1-3494-4dbf-bd94-ba7b6e9ed064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.550567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9f5f3bf1-3494-4dbf-bd94-ba7b6e9ed064 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:49.551785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9f5f3bf1-3494-4dbf-bd94-ba7b6e9ed064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:49.660116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:49.663967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 306/625] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:49.665556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f3bf1-3494-4dbf-bd94-ba7b6e9ed064 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:49.670192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e18d75-6050-4d8a-b7b1-5cedfd5f2553 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:49.735592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142b5a9-f258-4b4d-bb4a-fa7456fc038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:49.736131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142b5a9-f258-4b4d-bb4a-fa7456fc038c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.736408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142b5a9-f258-4b4d-bb4a-fa7456fc038c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.736678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142b5a9-f258-4b4d-bb4a-fa7456fc038c None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.737420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142b5a9-f258-4b4d-bb4a-fa7456fc038c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:49.737773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142b5a9-f258-4b4d-bb4a-fa7456fc038c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:49.739390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142b5a9-f258-4b4d-bb4a-fa7456fc038c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:49.765229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 320/626] 10.222.0.100 () {56 vars in 1140 bytes} [Thu Sep 1 17:23:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:49.766493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28808f5d-df1a-45d8-9c78-6b5a8b318009 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.766789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28808f5d-df1a-45d8-9c78-6b5a8b318009 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.767056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28808f5d-df1a-45d8-9c78-6b5a8b318009 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.768484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 321/627] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:49.769560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ca19c-b209-48cb-bb63-6b5c710bf140 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.770136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ca19c-b209-48cb-bb63-6b5c710bf140 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.770413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ca19c-b209-48cb-bb63-6b5c710bf140 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.772373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 322/628] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:49.773034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ca19c-b209-48cb-bb63-6b5c710bf140 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:49.782566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:49.786645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f3bf1-3494-4dbf-bd94-ba7b6e9ed064 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:49.826417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2f440-e953-41ea-9047-2a2956deabef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:49.827414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2f440-e953-41ea-9047-2a2956deabef None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.827842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2f440-e953-41ea-9047-2a2956deabef None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.828118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2f440-e953-41ea-9047-2a2956deabef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.839825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2f440-e953-41ea-9047-2a2956deabef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:49.841500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2f440-e953-41ea-9047-2a2956deabef None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a1a2f440-e953-41ea-9047-2a2956deabef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:49.856597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:49.857653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:49.858578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:49.858578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:49.858578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:49.858578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 307/629] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:49.859392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41119c94-92bc-4124-afbb-38102d6f273c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.859721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41119c94-92bc-4124-afbb-38102d6f273c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.860015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41119c94-92bc-4124-afbb-38102d6f273c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.880228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-41119c94-92bc-4124-afbb-38102d6f273c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:49.882254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-41119c94-92bc-4124-afbb-38102d6f273c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:49.893886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ca19c-b209-48cb-bb63-6b5c710bf140 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:49.932109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea17ce-9f03-4a71-aaeb-f9abaf533ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:49.932650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea17ce-9f03-4a71-aaeb-f9abaf533ffd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.933248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea17ce-9f03-4a71-aaeb-f9abaf533ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.933633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea17ce-9f03-4a71-aaeb-f9abaf533ffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.945674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea17ce-9f03-4a71-aaeb-f9abaf533ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:49.947254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea17ce-9f03-4a71-aaeb-f9abaf533ffd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:49.958350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 308/630] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:49.960492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a31a5-5443-46c9-99df-ced018031ce2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:49.960953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a31a5-5443-46c9-99df-ced018031ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:49.961395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a31a5-5443-46c9-99df-ced018031ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-ecea17ce-9f03-4a71-aaeb-f9abaf533ffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:49.963488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:49.964411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:49.965312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:49.965312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:49.965312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:49.969297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 323/631] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:49.970167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea17ce-9f03-4a71-aaeb-f9abaf533ffd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:49.983234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3f2a31a5-5443-46c9-99df-ced018031ce2 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:49.983650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:49.984609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3f2a31a5-5443-46c9-99df-ced018031ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:50.056573 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 309/632] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:50.057568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a31a5-5443-46c9-99df-ced018031ce2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.071910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:50.094235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea17ce-9f03-4a71-aaeb-f9abaf533ffd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.134495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2fab0-c1e6-478a-9044-73a8008d73bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.135247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2fab0-c1e6-478a-9044-73a8008d73bc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.135595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2fab0-c1e6-478a-9044-73a8008d73bc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.136116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2fab0-c1e6-478a-9044-73a8008d73bc None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.136629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2fab0-c1e6-478a-9044-73a8008d73bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:50.137028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2fab0-c1e6-478a-9044-73a8008d73bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:50.138520 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2fab0-c1e6-478a-9044-73a8008d73bc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:50.156690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 324/633] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:50.157704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2fab0-c1e6-478a-9044-73a8008d73bc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.193359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a31a5-5443-46c9-99df-ced018031ce2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.237960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:50.242049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380bd8d-fd42-4c32-b65c-47a9696414bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.242667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380bd8d-fd42-4c32-b65c-47a9696414bf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.242949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380bd8d-fd42-4c32-b65c-47a9696414bf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.243222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380bd8d-fd42-4c32-b65c-47a9696414bf None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.243869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380bd8d-fd42-4c32-b65c-47a9696414bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:50.244169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380bd8d-fd42-4c32-b65c-47a9696414bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:50.245535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380bd8d-fd42-4c32-b65c-47a9696414bf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:50.259403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 310/634] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:50.260269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380bd8d-fd42-4c32-b65c-47a9696414bf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.272602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2fab0-c1e6-478a-9044-73a8008d73bc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.311234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a411a-a9ff-42c4-886a-abd02f12b24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.311761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a411a-a9ff-42c4-886a-abd02f12b24e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.312036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a411a-a9ff-42c4-886a-abd02f12b24e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.312307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a411a-a9ff-42c4-886a-abd02f12b24e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.324394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a411a-a9ff-42c4-886a-abd02f12b24e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d95a411a-a9ff-42c4-886a-abd02f12b24e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:50.329070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:50.330044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:50.330878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 325/635] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:50.331613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a411a-a9ff-42c4-886a-abd02f12b24e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.348362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:50.379799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380bd8d-fd42-4c32-b65c-47a9696414bf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.414759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b697aa-457c-4a92-8fb0-0156debfb8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.415387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b697aa-457c-4a92-8fb0-0156debfb8ef None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.415667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b697aa-457c-4a92-8fb0-0156debfb8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.415939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b697aa-457c-4a92-8fb0-0156debfb8ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.422629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b697aa-457c-4a92-8fb0-0156debfb8ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:50.424067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b697aa-457c-4a92-8fb0-0156debfb8ef None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-31b697aa-457c-4a92-8fb0-0156debfb8ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:50.435109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:50.436080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:50.436966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:50.436966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:50.436966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:50.436966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:50.436966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:50.436966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:50.436966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:50.436966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:50.437628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a411a-a9ff-42c4-886a-abd02f12b24e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.437712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 311/636] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:50.437712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b697aa-457c-4a92-8fb0-0156debfb8ef None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.467116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:50.473447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63326e3e-6965-4b8c-804f-e7e7abc7d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.474103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63326e3e-6965-4b8c-804f-e7e7abc7d3ac None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.474384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63326e3e-6965-4b8c-804f-e7e7abc7d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.474656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63326e3e-6965-4b8c-804f-e7e7abc7d3ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.485441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63326e3e-6965-4b8c-804f-e7e7abc7d3ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-63326e3e-6965-4b8c-804f-e7e7abc7d3ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:50.490360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:50.491329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:50.492202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 326/637] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:50.492893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63326e3e-6965-4b8c-804f-e7e7abc7d3ac None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.546229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b697aa-457c-4a92-8fb0-0156debfb8ef None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.576791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:50.583305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f65252-98f0-437a-9e14-eba4bf9a6f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.583940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f65252-98f0-437a-9e14-eba4bf9a6f48 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.584818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f65252-98f0-437a-9e14-eba4bf9a6f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.585143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f65252-98f0-437a-9e14-eba4bf9a6f48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.592937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f65252-98f0-437a-9e14-eba4bf9a6f48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:50.594424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f65252-98f0-437a-9e14-eba4bf9a6f48 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:50.601455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63326e3e-6965-4b8c-804f-e7e7abc7d3ac None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-21f65252-98f0-437a-9e14-eba4bf9a6f48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:50.607778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:50.608958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:50.609999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:50.609999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:50.609999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:50.609999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:50.609999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:50.609999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:50.609999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:50.609999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:50.609999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 312/638] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:50.610714 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f65252-98f0-437a-9e14-eba4bf9a6f48 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.637017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b7d61-e8b3-4b1d-a2a2-88e270c67578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.637654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b7d61-e8b3-4b1d-a2a2-88e270c67578 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.637931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b7d61-e8b3-4b1d-a2a2-88e270c67578 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.638206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b7d61-e8b3-4b1d-a2a2-88e270c67578 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.638885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b7d61-e8b3-4b1d-a2a2-88e270c67578 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:50.639171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b7d61-e8b3-4b1d-a2a2-88e270c67578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:50.640551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b7d61-e8b3-4b1d-a2a2-88e270c67578 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:50.649302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 327/639] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:50.650165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b7d61-e8b3-4b1d-a2a2-88e270c67578 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.722262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:50.724446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f65252-98f0-437a-9e14-eba4bf9a6f48 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.759790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef197500-03d2-47c1-9211-26921ff70a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.760417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef197500-03d2-47c1-9211-26921ff70a9e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.760692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef197500-03d2-47c1-9211-26921ff70a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.760966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef197500-03d2-47c1-9211-26921ff70a9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.769487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b7d61-e8b3-4b1d-a2a2-88e270c67578 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.770940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef197500-03d2-47c1-9211-26921ff70a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-ef197500-03d2-47c1-9211-26921ff70a9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:50.777645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:50.778875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 313/640] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:50.780090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef197500-03d2-47c1-9211-26921ff70a9e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.806864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53888c6e-2337-4c1f-9b5e-41e928b06ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.807470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53888c6e-2337-4c1f-9b5e-41e928b06ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.807751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53888c6e-2337-4c1f-9b5e-41e928b06ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.808025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53888c6e-2337-4c1f-9b5e-41e928b06ad6 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.808717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53888c6e-2337-4c1f-9b5e-41e928b06ad6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:50.809020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53888c6e-2337-4c1f-9b5e-41e928b06ad6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:50.810466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53888c6e-2337-4c1f-9b5e-41e928b06ad6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:50.823337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 328/641] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:50.824260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53888c6e-2337-4c1f-9b5e-41e928b06ad6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.833817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:50.886116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef197500-03d2-47c1-9211-26921ff70a9e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.920669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca07a0-8494-4151-8da0-372fcf9102b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.921301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca07a0-8494-4151-8da0-372fcf9102b2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.921598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca07a0-8494-4151-8da0-372fcf9102b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.921884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca07a0-8494-4151-8da0-372fcf9102b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.922562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca07a0-8494-4151-8da0-372fcf9102b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:50.922867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca07a0-8494-4151-8da0-372fcf9102b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:50.924410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca07a0-8494-4151-8da0-372fcf9102b2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:50.930558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53888c6e-2337-4c1f-9b5e-41e928b06ad6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:50.939875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 314/642] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:50.939875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca07a0-8494-4151-8da0-372fcf9102b2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:50.947316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:50.969868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab9599-2754-4d68-8416-5bf549fe16bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:50.970467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab9599-2754-4d68-8416-5bf549fe16bb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:50.970843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab9599-2754-4d68-8416-5bf549fe16bb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:50.970988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab9599-2754-4d68-8416-5bf549fe16bb None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:50.971677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab9599-2754-4d68-8416-5bf549fe16bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:50.971974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab9599-2754-4d68-8416-5bf549fe16bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:50.973400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab9599-2754-4d68-8416-5bf549fe16bb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:50.985729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 329/643] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:50.986571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab9599-2754-4d68-8416-5bf549fe16bb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.043867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca07a0-8494-4151-8da0-372fcf9102b2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.057757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:51.081391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0a1fe-86ee-4158-bd08-b49302d8e61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.081740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0a1fe-86ee-4158-bd08-b49302d8e61e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.082021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0a1fe-86ee-4158-bd08-b49302d8e61e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.082296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0a1fe-86ee-4158-bd08-b49302d8e61e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.093531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab9599-2754-4d68-8416-5bf549fe16bb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.093801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0a1fe-86ee-4158-bd08-b49302d8e61e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.095261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0a1fe-86ee-4158-bd08-b49302d8e61e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-77e0a1fe-86ee-4158-bd08-b49302d8e61e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:51.112147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:51.113382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:51.114598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:51.114598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:51.114598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:51.114598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 315/644] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:51.114598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0a1fe-86ee-4158-bd08-b49302d8e61e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.135429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f6f44-4e75-4adc-84f6-9d0397559218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.136039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f6f44-4e75-4adc-84f6-9d0397559218 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.136331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f6f44-4e75-4adc-84f6-9d0397559218 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.136596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f6f44-4e75-4adc-84f6-9d0397559218 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.137277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f6f44-4e75-4adc-84f6-9d0397559218 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:51.137568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f6f44-4e75-4adc-84f6-9d0397559218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.138979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f6f44-4e75-4adc-84f6-9d0397559218 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:51.148278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 330/645] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:50 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:51.148965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f6f44-4e75-4adc-84f6-9d0397559218 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.188912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:51.238994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0a1fe-86ee-4158-bd08-b49302d8e61e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.283653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f6f44-4e75-4adc-84f6-9d0397559218 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.291581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec2376-4758-4a70-ab02-1db8172f1fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.292493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec2376-4758-4a70-ab02-1db8172f1fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.292939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec2376-4758-4a70-ab02-1db8172f1fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.293380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec2376-4758-4a70-ab02-1db8172f1fc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.294664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec2376-4758-4a70-ab02-1db8172f1fc5 None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:51.295178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec2376-4758-4a70-ab02-1db8172f1fc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.297462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec2376-4758-4a70-ab02-1db8172f1fc5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:51.307205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 316/646] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:51 2022] GET /identity/v3/role_assignments?user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 610 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:51.308123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec2376-4758-4a70-ab02-1db8172f1fc5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.336478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc85d41-1fbf-4604-8938-2eb33ca5d263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.337533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc85d41-1fbf-4604-8938-2eb33ca5d263 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.338042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc85d41-1fbf-4604-8938-2eb33ca5d263 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.338499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc85d41-1fbf-4604-8938-2eb33ca5d263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.355381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc85d41-1fbf-4604-8938-2eb33ca5d263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.356770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc85d41-1fbf-4604-8938-2eb33ca5d263 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:51.359469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5cc85d41-1fbf-4604-8938-2eb33ca5d263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:51.373014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:51.374166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:51.375079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:51.375079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:51.375079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:51.375079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 331/647] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:51.375303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc85d41-1fbf-4604-8938-2eb33ca5d263 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.456282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec2376-4758-4a70-ab02-1db8172f1fc5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.487237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:51.504212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b5655-a0ec-4e75-91ba-cbfeb3105ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.505101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc85d41-1fbf-4604-8938-2eb33ca5d263 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.505257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b5655-a0ec-4e75-91ba-cbfeb3105ba3 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.505257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b5655-a0ec-4e75-91ba-cbfeb3105ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.505439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b5655-a0ec-4e75-91ba-cbfeb3105ba3 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/f3fa2499f5594b7798f6fe4443a14f4c/roles/3e624c5fdefb4c03bd915bbc9d097212` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.525109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b5655-a0ec-4e75-91ba-cbfeb3105ba3 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=f3fa2499f5594b7798f6fe4443a14f4c, role_id=3e624c5fdefb4c03bd915bbc9d097212)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.526267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b5655-a0ec-4e75-91ba-cbfeb3105ba3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:51.543815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 317/648] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:23:51 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/f3fa2499f5594b7798f6fe4443a14f4c/roles/3e624c5fdefb4c03bd915bbc9d097212 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:23:51.544700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b5655-a0ec-4e75-91ba-cbfeb3105ba3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.546849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa8979-d1d4-4c4f-a141-8d11a4f09b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.547471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa8979-d1d4-4c4f-a141-8d11a4f09b50 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.547813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa8979-d1d4-4c4f-a141-8d11a4f09b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.548090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa8979-d1d4-4c4f-a141-8d11a4f09b50 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.548770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa8979-d1d4-4c4f-a141-8d11a4f09b50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:51.549090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa8979-d1d4-4c4f-a141-8d11a4f09b50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.550510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa8979-d1d4-4c4f-a141-8d11a4f09b50 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:51.569141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 332/649] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:51.569141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa8979-d1d4-4c4f-a141-8d11a4f09b50 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.691422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa8979-d1d4-4c4f-a141-8d11a4f09b50 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.693517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b5655-a0ec-4e75-91ba-cbfeb3105ba3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.697813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:51.738354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df64ee-2370-41d2-8a6b-d5e66ae277ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.739112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df64ee-2370-41d2-8a6b-d5e66ae277ec None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.739402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df64ee-2370-41d2-8a6b-d5e66ae277ec None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.739678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df64ee-2370-41d2-8a6b-d5e66ae277ec None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.740370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df64ee-2370-41d2-8a6b-d5e66ae277ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:51.740698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df64ee-2370-41d2-8a6b-d5e66ae277ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.743049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df64ee-2370-41d2-8a6b-d5e66ae277ec None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:51.754485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f6c54-f067-49df-922a-5ad9d7ebfb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.755194 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 333/650] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:51.755303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f6c54-f067-49df-922a-5ad9d7ebfb64 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.755597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f6c54-f067-49df-922a-5ad9d7ebfb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.755866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f6c54-f067-49df-922a-5ad9d7ebfb64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.756136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df64ee-2370-41d2-8a6b-d5e66ae277ec None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.768960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:51.777552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f6c54-f067-49df-922a-5ad9d7ebfb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.780300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f6c54-f067-49df-922a-5ad9d7ebfb64 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-7c7f6c54-f067-49df-922a-5ad9d7ebfb64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:51.800014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:51.801028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:51.802086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:51.802086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:51.802086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:51.802086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 318/651] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:51.803170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f6c54-f067-49df-922a-5ad9d7ebfb64 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.879382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df64ee-2370-41d2-8a6b-d5e66ae277ec None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.918849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb2c33-4819-4d70-babd-ce0ce5237206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.919414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb2c33-4819-4d70-babd-ce0ce5237206 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.919701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb2c33-4819-4d70-babd-ce0ce5237206 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.919973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb2c33-4819-4d70-babd-ce0ce5237206 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.928845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb2c33-4819-4d70-babd-ce0ce5237206 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.930318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb2c33-4819-4d70-babd-ce0ce5237206 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-bfeb2c33-4819-4d70-babd-ce0ce5237206 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:51.944273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:51.945903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:23:51.946853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:51.947383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f6c54-f067-49df-922a-5ad9d7ebfb64 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:51.947456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 334/652] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:51.947456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb2c33-4819-4d70-babd-ce0ce5237206 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:51.978917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:51.985158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a298161-13b0-4cb7-a804-59705922518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:51.985820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a298161-13b0-4cb7-a804-59705922518f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:51.986101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a298161-13b0-4cb7-a804-59705922518f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:51.986375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a298161-13b0-4cb7-a804-59705922518f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:51.995199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a298161-13b0-4cb7-a804-59705922518f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:51.996769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a298161-13b0-4cb7-a804-59705922518f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5a298161-13b0-4cb7-a804-59705922518f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:52.010053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:52.011054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:52.011969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 319/653] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:52.012680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a298161-13b0-4cb7-a804-59705922518f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:52.022928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:52.061007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb2c33-4819-4d70-babd-ce0ce5237206 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.099218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136af51-d347-44cf-9c87-0238d7b84da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:52.099859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136af51-d347-44cf-9c87-0238d7b84da4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:52.100126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136af51-d347-44cf-9c87-0238d7b84da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:52.100400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136af51-d347-44cf-9c87-0238d7b84da4 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:52.101056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136af51-d347-44cf-9c87-0238d7b84da4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:52.101357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136af51-d347-44cf-9c87-0238d7b84da4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:52.103015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136af51-d347-44cf-9c87-0238d7b84da4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:52.119945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 335/654] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:52.120640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136af51-d347-44cf-9c87-0238d7b84da4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:52.150107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a298161-13b0-4cb7-a804-59705922518f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.199719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54edc18-a1c7-446a-bad4-9b38deeef612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:52.200661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edc18-a1c7-446a-bad4-9b38deeef612 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:52.201163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edc18-a1c7-446a-bad4-9b38deeef612 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:52.201899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edc18-a1c7-446a-bad4-9b38deeef612 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:52.203434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54edc18-a1c7-446a-bad4-9b38deeef612 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:52.204101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54edc18-a1c7-446a-bad4-9b38deeef612 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:52.206118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54edc18-a1c7-446a-bad4-9b38deeef612 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:52.235366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 320/655] 10.222.0.100 () {56 vars in 1142 bytes} [Thu Sep 1 17:23:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:52.235366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54edc18-a1c7-446a-bad4-9b38deeef612 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:52.245121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136af51-d347-44cf-9c87-0238d7b84da4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.276141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:52.286945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb514c-38ca-4a44-a378-40b3ddb49a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:52.288012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb514c-38ca-4a44-a378-40b3ddb49a25 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:52.288430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb514c-38ca-4a44-a378-40b3ddb49a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:52.288800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb514c-38ca-4a44-a378-40b3ddb49a25 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:52.289669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb514c-38ca-4a44-a378-40b3ddb49a25 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:52.290369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb514c-38ca-4a44-a378-40b3ddb49a25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:52.293301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb514c-38ca-4a44-a378-40b3ddb49a25 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:52.302243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:52.319420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 336/656] 10.222.0.100 () {56 vars in 1142 bytes} [Thu Sep 1 17:23:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:52.320771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb514c-38ca-4a44-a378-40b3ddb49a25 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:52.376323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54edc18-a1c7-446a-bad4-9b38deeef612 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.417070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc311dc9-07c0-4092-a4c8-3447d10dd38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:52.417880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc311dc9-07c0-4092-a4c8-3447d10dd38c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:52.418246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc311dc9-07c0-4092-a4c8-3447d10dd38c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:52.418625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc311dc9-07c0-4092-a4c8-3447d10dd38c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:52.429305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc311dc9-07c0-4092-a4c8-3447d10dd38c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:52.431081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc311dc9-07c0-4092-a4c8-3447d10dd38c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cc311dc9-07c0-4092-a4c8-3447d10dd38c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:52.446241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:52.447436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:23:52.448583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:52.455262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 321/657] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:52.456249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc311dc9-07c0-4092-a4c8-3447d10dd38c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:52.463757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb514c-38ca-4a44-a378-40b3ddb49a25 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.502049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964334e-c547-4520-8e68-7d6a999125fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:52.502859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964334e-c547-4520-8e68-7d6a999125fd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:52.503232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964334e-c547-4520-8e68-7d6a999125fd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:52.503599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964334e-c547-4520-8e68-7d6a999125fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:52.560954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:52.578753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964334e-c547-4520-8e68-7d6a999125fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:52.580356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964334e-c547-4520-8e68-7d6a999125fd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:52.582501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc311dc9-07c0-4092-a4c8-3447d10dd38c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.593154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:52.623336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d94e5-8da4-4ee3-bc87-39874fd817f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:52.624225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d94e5-8da4-4ee3-bc87-39874fd817f0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:52.624767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d94e5-8da4-4ee3-bc87-39874fd817f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:52.625258 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d94e5-8da4-4ee3-bc87-39874fd817f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:52.697085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 337/658] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:52 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:52.698134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964334e-c547-4520-8e68-7d6a999125fd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:52.706949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d94e5-8da4-4ee3-bc87-39874fd817f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:52.708797 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d94e5-8da4-4ee3-bc87-39874fd817f0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:52.793393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:52.812685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964334e-c547-4520-8e68-7d6a999125fd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.835437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 322/659] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:52 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:52.837031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d94e5-8da4-4ee3-bc87-39874fd817f0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:52.848862 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:52.852521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be8d48-3967-4681-a463-4c88bf2eb2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:52.853230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be8d48-3967-4681-a463-4c88bf2eb2be None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:52.853602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be8d48-3967-4681-a463-4c88bf2eb2be None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:52.854027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be8d48-3967-4681-a463-4c88bf2eb2be None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:52.854750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be8d48-3967-4681-a463-4c88bf2eb2be None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:52.855123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be8d48-3967-4681-a463-4c88bf2eb2be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:52.856611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be8d48-3967-4681-a463-4c88bf2eb2be None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:52.879670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 338/660] 10.222.0.100 () {56 vars in 1144 bytes} [Thu Sep 1 17:23:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:52.880718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be8d48-3967-4681-a463-4c88bf2eb2be None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:52.955626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d94e5-8da4-4ee3-bc87-39874fd817f0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.992550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be8d48-3967-4681-a463-4c88bf2eb2be None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:52.996297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a391f9-2e30-4d43-b0be-428735da8233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:52.997161 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a391f9-2e30-4d43-b0be-428735da8233 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:52.997641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a391f9-2e30-4d43-b0be-428735da8233 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:52.998091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a391f9-2e30-4d43-b0be-428735da8233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.010902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a391f9-2e30-4d43-b0be-428735da8233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.012923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a391f9-2e30-4d43-b0be-428735da8233 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-75a391f9-2e30-4d43-b0be-428735da8233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:53.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:53.032970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:53.033902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:53.033902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:53.033902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:53.037937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 323/661] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:53.039655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a391f9-2e30-4d43-b0be-428735da8233 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:53.042425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba36d4-bf7c-4e6e-b0d4-90704a214c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:53.043190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba36d4-bf7c-4e6e-b0d4-90704a214c42 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.043587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba36d4-bf7c-4e6e-b0d4-90704a214c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.043952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba36d4-bf7c-4e6e-b0d4-90704a214c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.056860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba36d4-bf7c-4e6e-b0d4-90704a214c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.058680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba36d4-bf7c-4e6e-b0d4-90704a214c42 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.061065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-75ba36d4-bf7c-4e6e-b0d4-90704a214c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:53.078122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:53.079449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:53.080496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:53.080496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:53.080496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:53.080496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 339/662] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:53.081563 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba36d4-bf7c-4e6e-b0d4-90704a214c42 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:53.124590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:53.173719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a391f9-2e30-4d43-b0be-428735da8233 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:53.198291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba36d4-bf7c-4e6e-b0d4-90704a214c42 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:53.216034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabb235-53e0-41cd-9f21-a4542be8e54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:53.217093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabb235-53e0-41cd-9f21-a4542be8e54b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.217720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabb235-53e0-41cd-9f21-a4542be8e54b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.218202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabb235-53e0-41cd-9f21-a4542be8e54b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.235841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c806fb-5f8a-41d3-b762-d8082c805b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:53.236473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c806fb-5f8a-41d3-b762-d8082c805b70 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.236759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c806fb-5f8a-41d3-b762-d8082c805b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.237027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c806fb-5f8a-41d3-b762-d8082c805b70 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.237819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c806fb-5f8a-41d3-b762-d8082c805b70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:53.238586 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c806fb-5f8a-41d3-b762-d8082c805b70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.239937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c806fb-5f8a-41d3-b762-d8082c805b70 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.257990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 340/663] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:53.258899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c806fb-5f8a-41d3-b762-d8082c805b70 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:53.320490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabb235-53e0-41cd-9f21-a4542be8e54b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.324125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabb235-53e0-41cd-9f21-a4542be8e54b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.325017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:53.376306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c806fb-5f8a-41d3-b762-d8082c805b70 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:53.407844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:53.415255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595752c7-9c22-4fce-8cea-21f26f5cdc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:53.415867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595752c7-9c22-4fce-8cea-21f26f5cdc9c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.416143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595752c7-9c22-4fce-8cea-21f26f5cdc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.416413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595752c7-9c22-4fce-8cea-21f26f5cdc9c None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.417059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595752c7-9c22-4fce-8cea-21f26f5cdc9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:53.417357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595752c7-9c22-4fce-8cea-21f26f5cdc9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.418749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595752c7-9c22-4fce-8cea-21f26f5cdc9c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.432533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 341/664] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:53.433916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595752c7-9c22-4fce-8cea-21f26f5cdc9c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:53.463298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 324/665] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:53 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:53.464494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabb235-53e0-41cd-9f21-a4542be8e54b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:53.557448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595752c7-9c22-4fce-8cea-21f26f5cdc9c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:53.573642 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:53.580504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabb235-53e0-41cd-9f21-a4542be8e54b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:53.600845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f7914-3686-42cb-a894-9b90e791db51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:53.601490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f7914-3686-42cb-a894-9b90e791db51 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.601800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f7914-3686-42cb-a894-9b90e791db51 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.602075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f7914-3686-42cb-a894-9b90e791db51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.609639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f7914-3686-42cb-a894-9b90e791db51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.611118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f7914-3686-42cb-a894-9b90e791db51 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-922f7914-3686-42cb-a894-9b90e791db51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:53.622851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:53.623811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 342/666] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:53.624739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f7914-3686-42cb-a894-9b90e791db51 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:53.628884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bccd4-bd02-4b4b-b9fd-3c50c41814c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:53.629474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bccd4-bd02-4b4b-b9fd-3c50c41814c0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.629833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bccd4-bd02-4b4b-b9fd-3c50c41814c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.630150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bccd4-bd02-4b4b-b9fd-3c50c41814c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.638229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bccd4-bd02-4b4b-b9fd-3c50c41814c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.639693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bccd4-bd02-4b4b-b9fd-3c50c41814c0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-fc3bccd4-bd02-4b4b-b9fd-3c50c41814c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:53.652285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:53.653391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 325/667] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99e8de-5d2f-4550-bc3e-b075b5ae0800 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.656580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99e8de-5d2f-4550-bc3e-b075b5ae0800 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.657834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99e8de-5d2f-4550-bc3e-b075b5ae0800 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.657834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 326/668] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:53.658346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6af020-d7db-4edb-bec3-535b429d9f2c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.658623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6af020-d7db-4edb-bec3-535b429d9f2c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.658892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6af020-d7db-4edb-bec3-535b429d9f2c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.660464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 327/669] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:53.661126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6af020-d7db-4edb-bec3-535b429d9f2c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:53.691517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:53.736999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f7914-3686-42cb-a894-9b90e791db51 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:53.773692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae7e4b-1d81-4de7-aa45-1ff75f80e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:53.774335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae7e4b-1d81-4de7-aa45-1ff75f80e324 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.774618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae7e4b-1d81-4de7-aa45-1ff75f80e324 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.774892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae7e4b-1d81-4de7-aa45-1ff75f80e324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.786369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae7e4b-1d81-4de7-aa45-1ff75f80e324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.788031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae7e4b-1d81-4de7-aa45-1ff75f80e324 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e2ae7e4b-1d81-4de7-aa45-1ff75f80e324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:53.804031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:53.805667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:53.806868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:53.806868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:53.806868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:53.807091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6af020-d7db-4edb-bec3-535b429d9f2c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:53.807191 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 343/670] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:53.807191 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae7e4b-1d81-4de7-aa45-1ff75f80e324 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:53.842758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:53.846254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0a0ae-e9a2-4bad-a48e-c13af26e7029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:53.846868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0a0ae-e9a2-4bad-a48e-c13af26e7029 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.847148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0a0ae-e9a2-4bad-a48e-c13af26e7029 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.847421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0a0ae-e9a2-4bad-a48e-c13af26e7029 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.848110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0a0ae-e9a2-4bad-a48e-c13af26e7029 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:53.848414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0a0ae-e9a2-4bad-a48e-c13af26e7029 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.849939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0a0ae-e9a2-4bad-a48e-c13af26e7029 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.862933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 328/671] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:53.864219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15099e92-98bb-4538-8bc2-b6c4e89ae7d5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.864504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15099e92-98bb-4538-8bc2-b6c4e89ae7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.864781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15099e92-98bb-4538-8bc2-b6c4e89ae7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.885027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-15099e92-98bb-4538-8bc2-b6c4e89ae7d5 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:53.886275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-15099e92-98bb-4538-8bc2-b6c4e89ae7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:53.924406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae7e4b-1d81-4de7-aa45-1ff75f80e324 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:53.958247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 329/672] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:53.958985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad6289-455c-4ba8-acfa-72497464dc2c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.959301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad6289-455c-4ba8-acfa-72497464dc2c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.959574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad6289-455c-4ba8-acfa-72497464dc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.970502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc0e0b-499b-479f-9a48-64eca0c1ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:53.971576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc0e0b-499b-479f-9a48-64eca0c1ac4a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:53.971863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc0e0b-499b-479f-9a48-64eca0c1ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:53.972142 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc0e0b-499b-479f-9a48-64eca0c1ac4a None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:53.972858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc0e0b-499b-479f-9a48-64eca0c1ac4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:53.973161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc0e0b-499b-479f-9a48-64eca0c1ac4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:53.974651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc0e0b-499b-479f-9a48-64eca0c1ac4a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:53.978011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-57ad6289-455c-4ba8-acfa-72497464dc2c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:53.979207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-57ad6289-455c-4ba8-acfa-72497464dc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:53.982799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:53.993734 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 344/673] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:53.996056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc0e0b-499b-479f-9a48-64eca0c1ac4a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.054040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 330/674] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:54.055329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad6289-455c-4ba8-acfa-72497464dc2c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.145643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc0e0b-499b-479f-9a48-64eca0c1ac4a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:54.149978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:54.176982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad6289-455c-4ba8-acfa-72497464dc2c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:54.198715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086eb80-a353-4225-b70e-25202e76be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:54.199331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086eb80-a353-4225-b70e-25202e76be16 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:54.199609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086eb80-a353-4225-b70e-25202e76be16 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:54.199880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086eb80-a353-4225-b70e-25202e76be16 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:54.200530 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086eb80-a353-4225-b70e-25202e76be16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:54.200827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086eb80-a353-4225-b70e-25202e76be16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:54.202894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086eb80-a353-4225-b70e-25202e76be16 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:54.220687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 345/675] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:54.222062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086eb80-a353-4225-b70e-25202e76be16 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.224515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf388303-7e74-437f-a7f7-608969d8cc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:54.226312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf388303-7e74-437f-a7f7-608969d8cc12 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:54.226590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf388303-7e74-437f-a7f7-608969d8cc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:54.226867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf388303-7e74-437f-a7f7-608969d8cc12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:54.228723 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf388303-7e74-437f-a7f7-608969d8cc12 None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=e0891d81e6e24ad99a9b532059c8c2af) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:54.229048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf388303-7e74-437f-a7f7-608969d8cc12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:54.230544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf388303-7e74-437f-a7f7-608969d8cc12 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:54.238257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 331/676] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:54 2022] GET /identity/v3/role_assignments?user.id=e0891d81e6e24ad99a9b532059c8c2af&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 610 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:54.239519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf388303-7e74-437f-a7f7-608969d8cc12 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.243679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:54.364223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086eb80-a353-4225-b70e-25202e76be16 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:54.390329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf388303-7e74-437f-a7f7-608969d8cc12 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:54.407676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c4c34-80b3-4f89-b720-c91b01160d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:54.408217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c4c34-80b3-4f89-b720-c91b01160d82 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:54.408505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c4c34-80b3-4f89-b720-c91b01160d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:54.409466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c4c34-80b3-4f89-b720-c91b01160d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:54.409604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c4c34-80b3-4f89-b720-c91b01160d82 None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=1d47d8e089f446d3bb4b6609d7256ac2) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:54.409841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c4c34-80b3-4f89-b720-c91b01160d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:54.411286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c4c34-80b3-4f89-b720-c91b01160d82 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:54.419425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 346/677] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:54 2022] GET /identity/v3/role_assignments?user.id=1d47d8e089f446d3bb4b6609d7256ac2&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 610 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:54.420616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c4c34-80b3-4f89-b720-c91b01160d82 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.431791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fc2f2-157a-46c1-adfc-d749e95bcbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:54.432456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fc2f2-157a-46c1-adfc-d749e95bcbff None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:54.432747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fc2f2-157a-46c1-adfc-d749e95bcbff None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:54.433023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fc2f2-157a-46c1-adfc-d749e95bcbff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:54.439159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:54.442887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fc2f2-157a-46c1-adfc-d749e95bcbff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:54.445195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fc2f2-157a-46c1-adfc-d749e95bcbff None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d68fc2f2-157a-46c1-adfc-d749e95bcbff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:54.459390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:54.460405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:54.461482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:54.461482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:54.461482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:54.461482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:54.461482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:54.461482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:54.461482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:54.461482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:54.461482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 332/678] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:54.462584 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fc2f2-157a-46c1-adfc-d749e95bcbff None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.560079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:54.576604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c4c34-80b3-4f89-b720-c91b01160d82 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:54.620200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fc2f2-157a-46c1-adfc-d749e95bcbff None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:54.634926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfd94c-d53c-4624-a215-b2bc52e36ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:54.635725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfd94c-d53c-4624-a215-b2bc52e36ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:54.636839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfd94c-d53c-4624-a215-b2bc52e36ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:54.637200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfd94c-d53c-4624-a215-b2bc52e36ac1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:54.654273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfd94c-d53c-4624-a215-b2bc52e36ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-2ccfd94c-d53c-4624-a215-b2bc52e36ac1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:54.664553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:54.665547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:54.666978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 347/679] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:54.667694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfd94c-d53c-4624-a215-b2bc52e36ac1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.678301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db7365-2005-4b79-8703-6167be1e59f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:54.679386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db7365-2005-4b79-8703-6167be1e59f6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:54.679947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db7365-2005-4b79-8703-6167be1e59f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:54.680485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db7365-2005-4b79-8703-6167be1e59f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:54.695158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db7365-2005-4b79-8703-6167be1e59f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-16db7365-2005-4b79-8703-6167be1e59f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:23:54.702216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:54.703668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 333/680] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:23:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:54.704596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db7365-2005-4b79-8703-6167be1e59f6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.757951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:54.790999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfd94c-d53c-4624-a215-b2bc52e36ac1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:54.827747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738476d-de1f-4364-a7ce-56ef24de5111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:54.828532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738476d-de1f-4364-a7ce-56ef24de5111 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:54.828884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738476d-de1f-4364-a7ce-56ef24de5111 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:54.829275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738476d-de1f-4364-a7ce-56ef24de5111 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:54.830114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738476d-de1f-4364-a7ce-56ef24de5111 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:54.830535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738476d-de1f-4364-a7ce-56ef24de5111 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:54.832061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738476d-de1f-4364-a7ce-56ef24de5111 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:54.834912 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db7365-2005-4b79-8703-6167be1e59f6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:54.846860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 348/681] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:54.847837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738476d-de1f-4364-a7ce-56ef24de5111 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.908337 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65afade-4fd8-454d-8f7f-4ce51baabd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:54.909381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65afade-4fd8-454d-8f7f-4ce51baabd0a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:54.909945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65afade-4fd8-454d-8f7f-4ce51baabd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:54.910455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65afade-4fd8-454d-8f7f-4ce51baabd0a None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:54.911668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65afade-4fd8-454d-8f7f-4ce51baabd0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:54.912259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65afade-4fd8-454d-8f7f-4ce51baabd0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:54.914871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65afade-4fd8-454d-8f7f-4ce51baabd0a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:54.921947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:54.929911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 334/682] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:23:54 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:54.931005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65afade-4fd8-454d-8f7f-4ce51baabd0a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:54.958815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738476d-de1f-4364-a7ce-56ef24de5111 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:54.991721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:54.998204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858097d-e028-416e-b6e4-45b7cf7717ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:54.998950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858097d-e028-416e-b6e4-45b7cf7717ad None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:54.999722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858097d-e028-416e-b6e4-45b7cf7717ad None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:54.999722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858097d-e028-416e-b6e4-45b7cf7717ad None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.000414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858097d-e028-416e-b6e4-45b7cf7717ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:55.000785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858097d-e028-416e-b6e4-45b7cf7717ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:55.002252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858097d-e028-416e-b6e4-45b7cf7717ad None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:55.016086 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 349/683] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:23:54 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:55.017166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858097d-e028-416e-b6e4-45b7cf7717ad None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:55.055966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65afade-4fd8-454d-8f7f-4ce51baabd0a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:55.094112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837dc9-79f2-496e-a063-c4c30283eb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:55.095115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837dc9-79f2-496e-a063-c4c30283eb50 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.095545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837dc9-79f2-496e-a063-c4c30283eb50 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.095973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837dc9-79f2-496e-a063-c4c30283eb50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.096856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb837dc9-79f2-496e-a063-c4c30283eb50 None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=bbe180b45f2e49e2abbf6e833ce0d44c) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:55.097326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb837dc9-79f2-496e-a063-c4c30283eb50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:55.098970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb837dc9-79f2-496e-a063-c4c30283eb50 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:55.113730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 335/684] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:54 2022] GET /identity/v3/role_assignments?user.id=bbe180b45f2e49e2abbf6e833ce0d44c&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 610 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:55.114991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837dc9-79f2-496e-a063-c4c30283eb50 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:55.131436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858097d-e028-416e-b6e4-45b7cf7717ad None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:55.171203 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0618-7291-4e77-a7fd-4cda944b04e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:55.171914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0618-7291-4e77-a7fd-4cda944b04e8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.172269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0618-7291-4e77-a7fd-4cda944b04e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.172627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0618-7291-4e77-a7fd-4cda944b04e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.186136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc0618-7291-4e77-a7fd-4cda944b04e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:55.188054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc0618-7291-4e77-a7fd-4cda944b04e8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1bcc0618-7291-4e77-a7fd-4cda944b04e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:55.204317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:55.205532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:55.206680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:55.206680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:55.206680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:55.207758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 350/685] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:55.221609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0618-7291-4e77-a7fd-4cda944b04e8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:55.246061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:55.282321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:55.354990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837dc9-79f2-496e-a063-c4c30283eb50 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:55.358153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0618-7291-4e77-a7fd-4cda944b04e8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:55.433919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebd129-9fe0-47b1-9b12-5fbd4281166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:55.434459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebd129-9fe0-47b1-9b12-5fbd4281166e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.434744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebd129-9fe0-47b1-9b12-5fbd4281166e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.435619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebd129-9fe0-47b1-9b12-5fbd4281166e None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.435779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebd129-9fe0-47b1-9b12-5fbd4281166e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:55.436032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebd129-9fe0-47b1-9b12-5fbd4281166e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:55.437455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebd129-9fe0-47b1-9b12-5fbd4281166e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:55.437999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c04836-8b7f-49ea-929e-81a2c1061445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:55.438653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c04836-8b7f-49ea-929e-81a2c1061445 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.438936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c04836-8b7f-49ea-929e-81a2c1061445 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.439214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c04836-8b7f-49ea-929e-81a2c1061445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.455971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 351/686] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:55.457339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebd129-9fe0-47b1-9b12-5fbd4281166e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:55.469181 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c04836-8b7f-49ea-929e-81a2c1061445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:55.472094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c04836-8b7f-49ea-929e-81a2c1061445 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-77c04836-8b7f-49ea-929e-81a2c1061445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:55.496235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:55.497458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:55.498750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:55.498750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:55.498750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:55.500622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 336/687] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:55.511201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c04836-8b7f-49ea-929e-81a2c1061445 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:55.586393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebd129-9fe0-47b1-9b12-5fbd4281166e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:55.591569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:55.639317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f055144-365c-4506-b306-3085fc232605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:55.639986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f055144-365c-4506-b306-3085fc232605 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.640277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f055144-365c-4506-b306-3085fc232605 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.640556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f055144-365c-4506-b306-3085fc232605 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.666499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f055144-365c-4506-b306-3085fc232605 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:55.669702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f055144-365c-4506-b306-3085fc232605 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6f055144-365c-4506-b306-3085fc232605 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:55.687009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:55.688927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:23:55.690964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:55.699558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 352/688] 10.222.0.100 () {56 vars in 1098 bytes} [Thu Sep 1 17:23:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:55.701329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f055144-365c-4506-b306-3085fc232605 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:55.733655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c04836-8b7f-49ea-929e-81a2c1061445 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:55.734180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:55.804764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22326f6-308d-411a-a880-c126f228d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:55.805392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22326f6-308d-411a-a880-c126f228d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.805701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22326f6-308d-411a-a880-c126f228d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.805976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22326f6-308d-411a-a880-c126f228d0e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.806634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22326f6-308d-411a-a880-c126f228d0e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:55.806940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22326f6-308d-411a-a880-c126f228d0e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:55.808329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22326f6-308d-411a-a880-c126f228d0e2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:55.837318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 337/689] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:55.843765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22326f6-308d-411a-a880-c126f228d0e2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:55.905865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f055144-365c-4506-b306-3085fc232605 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:55.970211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984895f4-ef3e-4656-b78b-2d62bfe94c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:55.970759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984895f4-ef3e-4656-b78b-2d62bfe94c95 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:55.971039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984895f4-ef3e-4656-b78b-2d62bfe94c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:55.971315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984895f4-ef3e-4656-b78b-2d62bfe94c95 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:55.972079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984895f4-ef3e-4656-b78b-2d62bfe94c95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:55.972391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984895f4-ef3e-4656-b78b-2d62bfe94c95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:55.973899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984895f4-ef3e-4656-b78b-2d62bfe94c95 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:56.011178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 353/690] 10.222.0.100 () {56 vars in 1138 bytes} [Thu Sep 1 17:23:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:56.018507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984895f4-ef3e-4656-b78b-2d62bfe94c95 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:56.043155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22326f6-308d-411a-a880-c126f228d0e2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:56.073979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:56.113491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71249087-e3ae-43cd-8402-a590d5661fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:56.114167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71249087-e3ae-43cd-8402-a590d5661fff None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.114452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71249087-e3ae-43cd-8402-a590d5661fff None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.114729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71249087-e3ae-43cd-8402-a590d5661fff None admin] PATH_INFO: `/v3/users/swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.130636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71249087-e3ae-43cd-8402-a590d5661fff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:56.132276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71249087-e3ae-43cd-8402-a590d5661fff None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-71249087-e3ae-43cd-8402-a590d5661fff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:23:56.154284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:56.155547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:23:56.156678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:56.160246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 338/691] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:23:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:56.161659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71249087-e3ae-43cd-8402-a590d5661fff None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:56.199055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:56.258829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984895f4-ef3e-4656-b78b-2d62bfe94c95 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:56.340078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a4be8-f48f-496a-a55f-823eeeb47b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:56.340705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a4be8-f48f-496a-a55f-823eeeb47b4e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.340987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a4be8-f48f-496a-a55f-823eeeb47b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.341260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a4be8-f48f-496a-a55f-823eeeb47b4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.372263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71249087-e3ae-43cd-8402-a590d5661fff None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:56.434936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5b4ce-f6ef-468e-a120-5ff758d6e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:56.435563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5b4ce-f6ef-468e-a120-5ff758d6e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.435848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5b4ce-f6ef-468e-a120-5ff758d6e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.436125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5b4ce-f6ef-468e-a120-5ff758d6e6b2 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.436878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5b4ce-f6ef-468e-a120-5ff758d6e6b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:56.437185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5b4ce-f6ef-468e-a120-5ff758d6e6b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:56.438666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5b4ce-f6ef-468e-a120-5ff758d6e6b2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:56.481075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 339/692] 10.222.0.100 () {56 vars in 1140 bytes} [Thu Sep 1 17:23:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:56.487458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5b4ce-f6ef-468e-a120-5ff758d6e6b2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:56.509995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a4be8-f48f-496a-a55f-823eeeb47b4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:56.511398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a4be8-f48f-496a-a55f-823eeeb47b4e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:56.540209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:56.675598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:56.710629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5b4ce-f6ef-468e-a120-5ff758d6e6b2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:56.713671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 354/693] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:56 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:23:56.715450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b7d10-1337-49b4-b4f8-34af167b4613 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.715870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b7d10-1337-49b4-b4f8-34af167b4613 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.716144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b7d10-1337-49b4-b4f8-34af167b4613 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.718448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 355/694] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:56 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:56.719423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b7d10-1337-49b4-b4f8-34af167b4613 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:56.764578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32c200-6109-4b8c-9dba-3ea3453f4024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:56.765268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c200-6109-4b8c-9dba-3ea3453f4024 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.766646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c200-6109-4b8c-9dba-3ea3453f4024 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.766646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c200-6109-4b8c-9dba-3ea3453f4024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.857450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b7d10-1337-49b4-b4f8-34af167b4613 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:56.858723 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32c200-6109-4b8c-9dba-3ea3453f4024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:56.861340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32c200-6109-4b8c-9dba-3ea3453f4024 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:56.887924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:56.904694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16444a-070d-45db-bc16-a11a82fe2340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:56.905495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16444a-070d-45db-bc16-a11a82fe2340 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.905915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16444a-070d-45db-bc16-a11a82fe2340 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.906303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16444a-070d-45db-bc16-a11a82fe2340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.919976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16444a-070d-45db-bc16-a11a82fe2340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:56.921511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16444a-070d-45db-bc16-a11a82fe2340 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0c16444a-070d-45db-bc16-a11a82fe2340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:56.941322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:56.943345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:56.945107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:56.945107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:56.945107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:56.945107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 356/695] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:23:56.945107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145ad7-ab45-4b31-965a-897a17cf58ba None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:56.945107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145ad7-ab45-4b31-965a-897a17cf58ba None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:56.945107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145ad7-ab45-4b31-965a-897a17cf58ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:56.965162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5b145ad7-ab45-4b31-965a-897a17cf58ba None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:56.966393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5b145ad7-ab45-4b31-965a-897a17cf58ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:56.971120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.001620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 340/696] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:23:56 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:23:57.001620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f775aea-3be5-4554-8566-35bd5a4b55c1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.001620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f775aea-3be5-4554-8566-35bd5a4b55c1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.001620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f775aea-3be5-4554-8566-35bd5a4b55c1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.001620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 341/697] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:57.001620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f775aea-3be5-4554-8566-35bd5a4b55c1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.057854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 357/698] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:57.059407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145ad7-ab45-4b31-965a-897a17cf58ba None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.125127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f775aea-3be5-4554-8566-35bd5a4b55c1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.168730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736c559-6140-49c2-9641-d4a38eaa8029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.170274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736c559-6140-49c2-9641-d4a38eaa8029 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.170705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736c559-6140-49c2-9641-d4a38eaa8029 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.171392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736c559-6140-49c2-9641-d4a38eaa8029 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.172445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736c559-6140-49c2-9641-d4a38eaa8029 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:57.172897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736c559-6140-49c2-9641-d4a38eaa8029 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.175194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736c559-6140-49c2-9641-d4a38eaa8029 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.183066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145ad7-ab45-4b31-965a-897a17cf58ba None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.186984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.192806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 342/699] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:57.194355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2ee47-52e2-4216-919c-63a57b44dfc1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.194792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2ee47-52e2-4216-919c-63a57b44dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.195260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2ee47-52e2-4216-919c-63a57b44dfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.218393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-85b2ee47-52e2-4216-919c-63a57b44dfc1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:57.219942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-85b2ee47-52e2-4216-919c-63a57b44dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:57.232429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec5342-a7bb-47f7-b4d1-8e185fa881c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.233288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec5342-a7bb-47f7-b4d1-8e185fa881c3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.233747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec5342-a7bb-47f7-b4d1-8e185fa881c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.234167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec5342-a7bb-47f7-b4d1-8e185fa881c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.245721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec5342-a7bb-47f7-b4d1-8e185fa881c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.247451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec5342-a7bb-47f7-b4d1-8e185fa881c3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.257699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9fec5342-a7bb-47f7-b4d1-8e185fa881c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:57.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:57.268030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:57.269043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:57.269043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:57.269043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:57.275358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 358/700] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:57.276617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec5342-a7bb-47f7-b4d1-8e185fa881c3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.295498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 343/701] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:57.296598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2ee47-52e2-4216-919c-63a57b44dfc1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.401804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2ee47-52e2-4216-919c-63a57b44dfc1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.415112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec5342-a7bb-47f7-b4d1-8e185fa881c3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.444343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74848834-08f9-48e6-bd26-de25bcfc4c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.444960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74848834-08f9-48e6-bd26-de25bcfc4c09 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.445241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74848834-08f9-48e6-bd26-de25bcfc4c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.445513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74848834-08f9-48e6-bd26-de25bcfc4c09 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.446280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74848834-08f9-48e6-bd26-de25bcfc4c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:57.446581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74848834-08f9-48e6-bd26-de25bcfc4c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.448012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74848834-08f9-48e6-bd26-de25bcfc4c09 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.463432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 344/702] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:57.463935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b499010-10cb-4c69-95e9-f3058a3f09d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.464289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74848834-08f9-48e6-bd26-de25bcfc4c09 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.464887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b499010-10cb-4c69-95e9-f3058a3f09d2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.465275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b499010-10cb-4c69-95e9-f3058a3f09d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.465689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b499010-10cb-4c69-95e9-f3058a3f09d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.478470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b499010-10cb-4c69-95e9-f3058a3f09d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.479978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b499010-10cb-4c69-95e9-f3058a3f09d2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0b499010-10cb-4c69-95e9-f3058a3f09d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:57.493120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:57.494162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:57.495323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:57.495323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:57.495323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:57.495323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:57.495323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:57.495323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:57.495323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:57.495323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:57.499413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.499547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 359/703] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:57.500837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b499010-10cb-4c69-95e9-f3058a3f09d2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.522755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.575513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74848834-08f9-48e6-bd26-de25bcfc4c09 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.613597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839b21e-ed54-4655-b360-2316b5efc5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.614257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839b21e-ed54-4655-b360-2316b5efc5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.614538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839b21e-ed54-4655-b360-2316b5efc5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.614813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839b21e-ed54-4655-b360-2316b5efc5e3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.615394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b499010-10cb-4c69-95e9-f3058a3f09d2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.616033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839b21e-ed54-4655-b360-2316b5efc5e3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.617563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839b21e-ed54-4655-b360-2316b5efc5e3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5839b21e-ed54-4655-b360-2316b5efc5e3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:57.628834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:57.629916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:23:57.630762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:57.635303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 345/704] 10.222.0.100 () {56 vars in 1134 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:57.636029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839b21e-ed54-4655-b360-2316b5efc5e3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.658003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d4029-40e0-4750-a201-3d5d1d5abb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.658647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d4029-40e0-4750-a201-3d5d1d5abb9d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.658985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d4029-40e0-4750-a201-3d5d1d5abb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.659198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d4029-40e0-4750-a201-3d5d1d5abb9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.667059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d4029-40e0-4750-a201-3d5d1d5abb9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.668495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d4029-40e0-4750-a201-3d5d1d5abb9d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-434d4029-40e0-4750-a201-3d5d1d5abb9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:23:57.679759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:57.680813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:57.681954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:23:57.681954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:23:57.681954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:23:57.681954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:23:57.681954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:23:57.681954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:23:57.681954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:23:57.681954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:57.681954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 360/705] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:57.682516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badeeff-6b57-41f9-8ac8-d8e76e31fa13 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.682516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badeeff-6b57-41f9-8ac8-d8e76e31fa13 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.682689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badeeff-6b57-41f9-8ac8-d8e76e31fa13 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.685628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 361/706] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:23:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:23:57.685628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badeeff-6b57-41f9-8ac8-d8e76e31fa13 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.744931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.751903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839b21e-ed54-4655-b360-2316b5efc5e3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.767250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.788946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ec07-5bd6-43ff-914f-994bda0ee7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.789563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ec07-5bd6-43ff-914f-994bda0ee7cc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.789875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ec07-5bd6-43ff-914f-994bda0ee7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.790149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ec07-5bd6-43ff-914f-994bda0ee7cc None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.790861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7ec07-5bd6-43ff-914f-994bda0ee7cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:57.791162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7ec07-5bd6-43ff-914f-994bda0ee7cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.792621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7ec07-5bd6-43ff-914f-994bda0ee7cc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.793430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badeeff-6b57-41f9-8ac8-d8e76e31fa13 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.804621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 346/707] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:57.805290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ec07-5bd6-43ff-914f-994bda0ee7cc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.831962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6478221-1a46-4bb9-9dba-521cd9ace768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.832567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6478221-1a46-4bb9-9dba-521cd9ace768 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.832845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6478221-1a46-4bb9-9dba-521cd9ace768 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.833111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6478221-1a46-4bb9-9dba-521cd9ace768 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.833767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6478221-1a46-4bb9-9dba-521cd9ace768 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:57.834646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6478221-1a46-4bb9-9dba-521cd9ace768 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.835962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6478221-1a46-4bb9-9dba-521cd9ace768 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.840684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 362/708] 10.222.0.100 () {56 vars in 1129 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:57.841303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6478221-1a46-4bb9-9dba-521cd9ace768 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:57.907178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ec07-5bd6-43ff-914f-994bda0ee7cc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.941055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b3724-53b1-4813-9b90-e5ed28ba9572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.941718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b3724-53b1-4813-9b90-e5ed28ba9572 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.942000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b3724-53b1-4813-9b90-e5ed28ba9572 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.942272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b3724-53b1-4813-9b90-e5ed28ba9572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.943702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6478221-1a46-4bb9-9dba-521cd9ace768 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:57.952915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b3724-53b1-4813-9b90-e5ed28ba9572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.954425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b3724-53b1-4813-9b90-e5ed28ba9572 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-263b3724-53b1-4813-9b90-e5ed28ba9572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:23:57.968959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:57.970128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:23:57.971231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:23:57.971231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:23:57.971231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:23:57.971231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 347/709] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:57.971231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718f548-fa29-4ae9-a372-a068f8ed7a6d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.971596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718f548-fa29-4ae9-a372-a068f8ed7a6d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.971774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718f548-fa29-4ae9-a372-a068f8ed7a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.982453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81d6b7-8e4d-4ef1-9aea-0a09a5b6cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:57.983063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81d6b7-8e4d-4ef1-9aea-0a09a5b6cd62 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:57.983340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81d6b7-8e4d-4ef1-9aea-0a09a5b6cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:57.983721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81d6b7-8e4d-4ef1-9aea-0a09a5b6cd62 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:57.984310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81d6b7-8e4d-4ef1-9aea-0a09a5b6cd62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:57.984620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81d6b7-8e4d-4ef1-9aea-0a09a5b6cd62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:57.985314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.986183 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81d6b7-8e4d-4ef1-9aea-0a09a5b6cd62 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:57.991191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9718f548-fa29-4ae9-a372-a068f8ed7a6d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:23:57.992470 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9718f548-fa29-4ae9-a372-a068f8ed7a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:23:57.995273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:57.999067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 363/710] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:58.000057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81d6b7-8e4d-4ef1-9aea-0a09a5b6cd62 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:58.061764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 348/711] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:23:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:23:58.062688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718f548-fa29-4ae9-a372-a068f8ed7a6d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:58.108372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81d6b7-8e4d-4ef1-9aea-0a09a5b6cd62 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:58.152061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91db3e4-853c-45cd-9a20-119bbdaeea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:58.152921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91db3e4-853c-45cd-9a20-119bbdaeea6c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.153198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91db3e4-853c-45cd-9a20-119bbdaeea6c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.153467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91db3e4-853c-45cd-9a20-119bbdaeea6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.154646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91db3e4-853c-45cd-9a20-119bbdaeea6c None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:58.154962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91db3e4-853c-45cd-9a20-119bbdaeea6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:58.156889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91db3e4-853c-45cd-9a20-119bbdaeea6c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:58.173693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 364/712] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:57 2022] GET /identity/v3/role_assignments?user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:58.173693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91db3e4-853c-45cd-9a20-119bbdaeea6c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:58.174030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718f548-fa29-4ae9-a372-a068f8ed7a6d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:58.211893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e18a9a-9f74-4699-b02a-c8b0797e4323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:58.212674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e18a9a-9f74-4699-b02a-c8b0797e4323 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.212956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e18a9a-9f74-4699-b02a-c8b0797e4323 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.213371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e18a9a-9f74-4699-b02a-c8b0797e4323 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.214044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e18a9a-9f74-4699-b02a-c8b0797e4323 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:58.214349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e18a9a-9f74-4699-b02a-c8b0797e4323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:58.215752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e18a9a-9f74-4699-b02a-c8b0797e4323 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:58.219844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:58.223458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 349/713] 10.222.0.100 () {56 vars in 1129 bytes} [Thu Sep 1 17:23:58 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:58.224626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e18a9a-9f74-4699-b02a-c8b0797e4323 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:58.264612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:58.352168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91db3e4-853c-45cd-9a20-119bbdaeea6c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:58.381883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e18a9a-9f74-4699-b02a-c8b0797e4323 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:58.404591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e308dc-30cf-4d85-b8d4-d4b4f57101be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:58.405565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e308dc-30cf-4d85-b8d4-d4b4f57101be None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.405872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e308dc-30cf-4d85-b8d4-d4b4f57101be None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.406145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e308dc-30cf-4d85-b8d4-d4b4f57101be None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.406966 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e308dc-30cf-4d85-b8d4-d4b4f57101be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:58.407497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e308dc-30cf-4d85-b8d4-d4b4f57101be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:58.409537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e308dc-30cf-4d85-b8d4-d4b4f57101be None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:58.427254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877a5bc-beda-4e1a-826b-1229412b5aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:58.427836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877a5bc-beda-4e1a-826b-1229412b5aaf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.428170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 365/714] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:23:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:58.429127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e308dc-30cf-4d85-b8d4-d4b4f57101be None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:58.432837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877a5bc-beda-4e1a-826b-1229412b5aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.433128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877a5bc-beda-4e1a-826b-1229412b5aaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.433936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877a5bc-beda-4e1a-826b-1229412b5aaf None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=f3fa2499f5594b7798f6fe4443a14f4c) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:58.434273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877a5bc-beda-4e1a-826b-1229412b5aaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:58.437225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877a5bc-beda-4e1a-826b-1229412b5aaf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:58.444567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 350/715] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:23:58 2022] GET /identity/v3/role_assignments?user.id=f3fa2499f5594b7798f6fe4443a14f4c&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 610 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:23:58.445624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877a5bc-beda-4e1a-826b-1229412b5aaf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:58.546018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:58.572075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e308dc-30cf-4d85-b8d4-d4b4f57101be None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:58.592891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877a5bc-beda-4e1a-826b-1229412b5aaf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:58.603462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:58.618373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6738c5-0b6b-48cc-9f5e-71fdaf4f86d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:58.619011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6738c5-0b6b-48cc-9f5e-71fdaf4f86d1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.619295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6738c5-0b6b-48cc-9f5e-71fdaf4f86d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.619578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6738c5-0b6b-48cc-9f5e-71fdaf4f86d1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.620457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6738c5-0b6b-48cc-9f5e-71fdaf4f86d1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:58.621873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6738c5-0b6b-48cc-9f5e-71fdaf4f86d1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-aa6738c5-0b6b-48cc-9f5e-71fdaf4f86d1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:23:58.637369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:23:58.638473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:23:58.639299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 366/716] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:23:58 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:23:58.644422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0633f3-8622-4e45-8e73-06b866378613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:58.644581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6738c5-0b6b-48cc-9f5e-71fdaf4f86d1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:58.645468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0633f3-8622-4e45-8e73-06b866378613 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.645990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0633f3-8622-4e45-8e73-06b866378613 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.646488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0633f3-8622-4e45-8e73-06b866378613 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.647866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0633f3-8622-4e45-8e73-06b866378613 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:58.653681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0633f3-8622-4e45-8e73-06b866378613 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:58.688147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 351/717] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:23:58 2022] POST /identity/v3/users => generated 255 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:23:58.779821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6738c5-0b6b-48cc-9f5e-71fdaf4f86d1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:58.819393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6983713-fb65-46ed-bf73-05bea1f6256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:58.820023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983713-fb65-46ed-bf73-05bea1f6256a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.820303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983713-fb65-46ed-bf73-05bea1f6256a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.820574 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983713-fb65-46ed-bf73-05bea1f6256a None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.821223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6983713-fb65-46ed-bf73-05bea1f6256a None admin] RBAC: Adding query filter params (name=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:58.821525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6983713-fb65-46ed-bf73-05bea1f6256a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:58.822992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6983713-fb65-46ed-bf73-05bea1f6256a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:58.828656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 367/718] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:23:58 2022] GET /identity/v3/services?name=network => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:23:58.834628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0633f3-8622-4e45-8e73-06b866378613 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:23:58.949654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0633f3-8622-4e45-8e73-06b866378613 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:23:58.971305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:23:58.987992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f680a36-8b6e-42f4-ac06-dbec2d853c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:23:58.988607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f680a36-8b6e-42f4-ac06-dbec2d853c6e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:23:58.988938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f680a36-8b6e-42f4-ac06-dbec2d853c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:23:58.989216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f680a36-8b6e-42f4-ac06-dbec2d853c6e None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:23:58.989884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f680a36-8b6e-42f4-ac06-dbec2d853c6e None admin] RBAC: Adding query filter params (type=network) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:23:58.990238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f680a36-8b6e-42f4-ac06-dbec2d853c6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:23:58.991743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f680a36-8b6e-42f4-ac06-dbec2d853c6e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:23:58.996854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 352/719] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:23:58 2022] GET /identity/v3/services?type=network => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:00.409889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd8773-90ba-4f41-b0a6-06b4d8a6df03 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:00.410418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd8773-90ba-4f41-b0a6-06b4d8a6df03 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:00.410418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd8773-90ba-4f41-b0a6-06b4d8a6df03 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:00.412025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 368/720] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:00.418400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27447502-51ec-471f-af16-3d475d5cfaaa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:00.418860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27447502-51ec-471f-af16-3d475d5cfaaa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:00.418941 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27447502-51ec-471f-af16-3d475d5cfaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:00.450852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-27447502-51ec-471f-af16-3d475d5cfaaa None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:00.452159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-27447502-51ec-471f-af16-3d475d5cfaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:00.542382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 353/721] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:24:00.787487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa39b1-f980-46ea-9a03-259a5e167234 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:00.787669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa39b1-f980-46ea-9a03-259a5e167234 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:00.787952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa39b1-f980-46ea-9a03-259a5e167234 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:00.789593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 369/722] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:00.802681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b94ad-b8dd-42d9-a697-51b6b84fff88 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:00.803818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b94ad-b8dd-42d9-a697-51b6b84fff88 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:00.804366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b94ad-b8dd-42d9-a697-51b6b84fff88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:00.826858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-851b94ad-b8dd-42d9-a697-51b6b84fff88 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:00.828389 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-851b94ad-b8dd-42d9-a697-51b6b84fff88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:00.912334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 354/723] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:24:00.989490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa39b1-f980-46ea-9a03-259a5e167234 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:01.049142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e4ff8-6f75-43ee-8fff-dbef2a32b173 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.049815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e4ff8-6f75-43ee-8fff-dbef2a32b173 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.050293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e4ff8-6f75-43ee-8fff-dbef2a32b173 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.052073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 355/724] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:01.057374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc006da-3841-4c9f-9ece-d2b5768efedf None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.057957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc006da-3841-4c9f-9ece-d2b5768efedf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.058431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc006da-3841-4c9f-9ece-d2b5768efedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.062335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:01.086081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cdc006da-3841-4c9f-9ece-d2b5768efedf None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:01.087296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cdc006da-3841-4c9f-9ece-d2b5768efedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:01.152763 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa39b1-f980-46ea-9a03-259a5e167234 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:01.163040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:01.213925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 356/725] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:24:01.225105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77181f87-71f5-4367-ae5a-d6710caa5f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:01.225781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181f87-71f5-4367-ae5a-d6710caa5f1f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.226066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181f87-71f5-4367-ae5a-d6710caa5f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.226338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181f87-71f5-4367-ae5a-d6710caa5f1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.245305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77181f87-71f5-4367-ae5a-d6710caa5f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-77181f87-71f5-4367-ae5a-d6710caa5f1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:01.250609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:01.251561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:01.252409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:01.253134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 370/726] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:01.258675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc006da-3841-4c9f-9ece-d2b5768efedf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:01.268696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77181f87-71f5-4367-ae5a-d6710caa5f1f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:01.447291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc006da-3841-4c9f-9ece-d2b5768efedf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:01.456317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77181f87-71f5-4367-ae5a-d6710caa5f1f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:01.469785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:01.488747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffcdc9-db05-4c5a-937d-a1939afc9606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:01.489354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffcdc9-db05-4c5a-937d-a1939afc9606 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.489658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffcdc9-db05-4c5a-937d-a1939afc9606 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.489933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffcdc9-db05-4c5a-937d-a1939afc9606 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.491226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffcdc9-db05-4c5a-937d-a1939afc9606 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:01.491709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffcdc9-db05-4c5a-937d-a1939afc9606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:01.493086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffcdc9-db05-4c5a-937d-a1939afc9606 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:01.495744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0ca1e-5ad0-4beb-8da5-d9c43d9ad442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:01.496555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0ca1e-5ad0-4beb-8da5-d9c43d9ad442 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.497044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0ca1e-5ad0-4beb-8da5-d9c43d9ad442 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.497499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0ca1e-5ad0-4beb-8da5-d9c43d9ad442 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.498837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0ca1e-5ad0-4beb-8da5-d9c43d9ad442 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:01.500736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0ca1e-5ad0-4beb-8da5-d9c43d9ad442 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:01.501136 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 357/727] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:24:01 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:01.502336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df4924-0abc-4542-a014-098c36dba7ee None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.502625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df4924-0abc-4542-a014-098c36dba7ee None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.502899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df4924-0abc-4542-a014-098c36dba7ee None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.506072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 358/728] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:01.509019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae045e-f89d-408c-afef-54f108ae87e3 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.509019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae045e-f89d-408c-afef-54f108ae87e3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.509299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae045e-f89d-408c-afef-54f108ae87e3 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.511262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 359/729] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:01.514509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae045e-f89d-408c-afef-54f108ae87e3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:01.525317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 371/730] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:24:01 2022] POST /identity/v3/services => generated 252 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:01.527627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1476d7-56d1-439e-8b32-1572d13a0457 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.528099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1476d7-56d1-439e-8b32-1572d13a0457 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.528877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1476d7-56d1-439e-8b32-1572d13a0457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.552358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dc1476d7-56d1-439e-8b32-1572d13a0457 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:01.554646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dc1476d7-56d1-439e-8b32-1572d13a0457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:01.556547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:01.649347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae045e-f89d-408c-afef-54f108ae87e3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:01.666263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 372/731] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:01 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:24:01.668267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e158bc-a459-4bfe-8fd9-7446822bf4ae None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.668716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e158bc-a459-4bfe-8fd9-7446822bf4ae None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.669631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e158bc-a459-4bfe-8fd9-7446822bf4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.690427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-14e158bc-a459-4bfe-8fd9-7446822bf4ae None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:01.692167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-14e158bc-a459-4bfe-8fd9-7446822bf4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:01.696296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e55d7-cdcd-491e-916e-721e7a9c1760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:01.696946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e55d7-cdcd-491e-916e-721e7a9c1760 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.697222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e55d7-cdcd-491e-916e-721e7a9c1760 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.697555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e55d7-cdcd-491e-916e-721e7a9c1760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.710854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e55d7-cdcd-491e-916e-721e7a9c1760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:01.712310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e55d7-cdcd-491e-916e-721e7a9c1760 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-357e55d7-cdcd-491e-916e-721e7a9c1760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:01.732873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:01.733946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:01.734829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:01.734829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:01.734829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:01.737667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 360/732] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:01.738647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e55d7-cdcd-491e-916e-721e7a9c1760 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:01.789928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 373/733] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:01 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:24:01.791571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e158bc-a459-4bfe-8fd9-7446822bf4ae None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:01.830886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:01.840691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:01.861206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e55d7-cdcd-491e-916e-721e7a9c1760 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:01.916787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf6cda-32d2-4dee-a20f-f23fe08a714b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:01.917516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf6cda-32d2-4dee-a20f-f23fe08a714b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.919035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf6cda-32d2-4dee-a20f-f23fe08a714b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.919307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf6cda-32d2-4dee-a20f-f23fe08a714b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.933238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e158bc-a459-4bfe-8fd9-7446822bf4ae None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:01.941833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf6cda-32d2-4dee-a20f-f23fe08a714b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-00cf6cda-32d2-4dee-a20f-f23fe08a714b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:01.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:01.948976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 361/734] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:01.950509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf6cda-32d2-4dee-a20f-f23fe08a714b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:01.983637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa6a3-a6e7-4326-9746-a4c2a21ed4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:01.984541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa6a3-a6e7-4326-9746-a4c2a21ed4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:01.984815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa6a3-a6e7-4326-9746-a4c2a21ed4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:01.985082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa6a3-a6e7-4326-9746-a4c2a21ed4c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:01.986017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaa6a3-a6e7-4326-9746-a4c2a21ed4c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:01.986706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaa6a3-a6e7-4326-9746-a4c2a21ed4c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:01.988492 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaa6a3-a6e7-4326-9746-a4c2a21ed4c4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.003681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 374/735] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:02.004637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa6a3-a6e7-4326-9746-a4c2a21ed4c4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.075513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf6cda-32d2-4dee-a20f-f23fe08a714b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.106590 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:02.125245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa6a3-a6e7-4326-9746-a4c2a21ed4c4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.134396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0588b-5782-44cf-a050-6482e3c37a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.135000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0588b-5782-44cf-a050-6482e3c37a43 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.135635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0588b-5782-44cf-a050-6482e3c37a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.135912 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0588b-5782-44cf-a050-6482e3c37a43 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:02.136899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0588b-5782-44cf-a050-6482e3c37a43 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:02.138298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0588b-5782-44cf-a050-6482e3c37a43 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.155554 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 362/736] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:24:01 2022] POST /identity/v3/services => generated 241 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:02.156208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0588b-5782-44cf-a050-6482e3c37a43 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.178520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:02.180633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42593bcc-0d1c-49da-9c61-5d66ec108bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.181603 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42593bcc-0d1c-49da-9c61-5d66ec108bec None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.182132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42593bcc-0d1c-49da-9c61-5d66ec108bec None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.182613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42593bcc-0d1c-49da-9c61-5d66ec108bec None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:02.184178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42593bcc-0d1c-49da-9c61-5d66ec108bec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:02.184725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42593bcc-0d1c-49da-9c61-5d66ec108bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:02.186854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42593bcc-0d1c-49da-9c61-5d66ec108bec None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.201038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 375/737] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:02.202244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42593bcc-0d1c-49da-9c61-5d66ec108bec None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.298493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0588b-5782-44cf-a050-6482e3c37a43 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.331628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42593bcc-0d1c-49da-9c61-5d66ec108bec None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.344048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feee2d8-4946-4002-a58b-fadca55af0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.344831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feee2d8-4946-4002-a58b-fadca55af0fb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.345110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feee2d8-4946-4002-a58b-fadca55af0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.345382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feee2d8-4946-4002-a58b-fadca55af0fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:02.357142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feee2d8-4946-4002-a58b-fadca55af0fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:02.359993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feee2d8-4946-4002-a58b-fadca55af0fb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-3feee2d8-4946-4002-a58b-fadca55af0fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:02.371792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:02.372803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:02.373799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 363/738] 10.222.0.100 () {56 vars in 1098 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:02.374459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feee2d8-4946-4002-a58b-fadca55af0fb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.375534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1851e7-1ae3-4f52-9f49-07d86a62b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.376162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1851e7-1ae3-4f52-9f49-07d86a62b94c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.376436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1851e7-1ae3-4f52-9f49-07d86a62b94c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.376698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1851e7-1ae3-4f52-9f49-07d86a62b94c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:02.394018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1851e7-1ae3-4f52-9f49-07d86a62b94c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-fc1851e7-1ae3-4f52-9f49-07d86a62b94c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:02.402129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:02.403725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:02.404711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 376/739] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:02.405597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1851e7-1ae3-4f52-9f49-07d86a62b94c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.421842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:02.468074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:02.513951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feee2d8-4946-4002-a58b-fadca55af0fb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.550578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1851e7-1ae3-4f52-9f49-07d86a62b94c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.551873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176cb85-31d2-402c-8f42-6327012b17c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.552721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176cb85-31d2-402c-8f42-6327012b17c6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.553417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176cb85-31d2-402c-8f42-6327012b17c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.554038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176cb85-31d2-402c-8f42-6327012b17c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:02.565513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176cb85-31d2-402c-8f42-6327012b17c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:02.567384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176cb85-31d2-402c-8f42-6327012b17c6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a176cb85-31d2-402c-8f42-6327012b17c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:02.583344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:02.585002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:02.586232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:02.586232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:02.586232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:02.588790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 364/740] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:02.590096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176cb85-31d2-402c-8f42-6327012b17c6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.591104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3c53-14cb-4bfa-b2a9-253314131291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.591799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3c53-14cb-4bfa-b2a9-253314131291 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.592087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3c53-14cb-4bfa-b2a9-253314131291 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.592360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3c53-14cb-4bfa-b2a9-253314131291 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:02.593101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3c53-14cb-4bfa-b2a9-253314131291 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:02.593405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3c53-14cb-4bfa-b2a9-253314131291 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:02.595246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3c53-14cb-4bfa-b2a9-253314131291 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.621745 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 377/741] 10.222.0.100 () {56 vars in 1138 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:02.622969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3c53-14cb-4bfa-b2a9-253314131291 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.714429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:02.715866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176cb85-31d2-402c-8f42-6327012b17c6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.732862 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:02.751674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84322d46-df15-43a9-b959-2dbe54ca9fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.752543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84322d46-df15-43a9-b959-2dbe54ca9fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.752543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84322d46-df15-43a9-b959-2dbe54ca9fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.752891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84322d46-df15-43a9-b959-2dbe54ca9fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:02.753502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84322d46-df15-43a9-b959-2dbe54ca9fb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:02.753837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84322d46-df15-43a9-b959-2dbe54ca9fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:02.755215 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84322d46-df15-43a9-b959-2dbe54ca9fb0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.764413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3c53-14cb-4bfa-b2a9-253314131291 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.769267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 365/742] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:02.770222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84322d46-df15-43a9-b959-2dbe54ca9fb0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.806555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef532b-3c6d-4501-9565-64b348e95376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.807458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef532b-3c6d-4501-9565-64b348e95376 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.807880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef532b-3c6d-4501-9565-64b348e95376 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.808395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef532b-3c6d-4501-9565-64b348e95376 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:02.809257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef532b-3c6d-4501-9565-64b348e95376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:02.809770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef532b-3c6d-4501-9565-64b348e95376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:02.811373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef532b-3c6d-4501-9565-64b348e95376 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.826235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 378/743] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:02.827326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef532b-3c6d-4501-9565-64b348e95376 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.878925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84322d46-df15-43a9-b959-2dbe54ca9fb0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.915016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfba56-04a0-4c48-a677-b8348d911364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.915654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfba56-04a0-4c48-a677-b8348d911364 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.915936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfba56-04a0-4c48-a677-b8348d911364 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.916214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfba56-04a0-4c48-a677-b8348d911364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:02.927027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfba56-04a0-4c48-a677-b8348d911364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:02.929101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfba56-04a0-4c48-a677-b8348d911364 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:02.943333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef532b-3c6d-4501-9565-64b348e95376 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-bddfba56-04a0-4c48-a677-b8348d911364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:02.944280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:02.945680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:02.946591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:02.946591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:02.946591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:02.946591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 366/744] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:02.946821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfba56-04a0-4c48-a677-b8348d911364 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:02.972991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:02.978326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:02.987815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26769-ec1a-491a-84df-79d613acc047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:02.988721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26769-ec1a-491a-84df-79d613acc047 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:02.989242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26769-ec1a-491a-84df-79d613acc047 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:02.989696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26769-ec1a-491a-84df-79d613acc047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.002676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e26769-ec1a-491a-84df-79d613acc047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.004433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e26769-ec1a-491a-84df-79d613acc047 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-86e26769-ec1a-491a-84df-79d613acc047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:03.021046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:03.022620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:03.023649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:03.023649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:03.023649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:03.025359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 379/745] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:03.026362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26769-ec1a-491a-84df-79d613acc047 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:03.069939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfba56-04a0-4c48-a677-b8348d911364 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:03.112402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034469f-c71b-4ccf-844a-b05f90db0eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:03.113345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034469f-c71b-4ccf-844a-b05f90db0eca None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.113774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034469f-c71b-4ccf-844a-b05f90db0eca None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.114968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034469f-c71b-4ccf-844a-b05f90db0eca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.123042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034469f-c71b-4ccf-844a-b05f90db0eca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.124763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034469f-c71b-4ccf-844a-b05f90db0eca None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5034469f-c71b-4ccf-844a-b05f90db0eca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:03.138405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:03.139745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:24:03.140996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:03.142778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 367/746] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:03.143834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034469f-c71b-4ccf-844a-b05f90db0eca None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:03.158478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26769-ec1a-491a-84df-79d613acc047 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:03.202052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd9bb2-c887-4f05-a045-4477712df7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:03.202801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd9bb2-c887-4f05-a045-4477712df7df None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.203241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd9bb2-c887-4f05-a045-4477712df7df None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.203613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd9bb2-c887-4f05-a045-4477712df7df None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.204368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd9bb2-c887-4f05-a045-4477712df7df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:03.204780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd9bb2-c887-4f05-a045-4477712df7df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.206265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd9bb2-c887-4f05-a045-4477712df7df None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.229962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 380/747] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:03.231282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd9bb2-c887-4f05-a045-4477712df7df None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:03.258076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:03.269428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034469f-c71b-4ccf-844a-b05f90db0eca None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:03.282944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:03.308009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e7314-1112-49c2-8efd-9e19ca1e3ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:03.308764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e7314-1112-49c2-8efd-9e19ca1e3ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.309138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e7314-1112-49c2-8efd-9e19ca1e3ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.309503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e7314-1112-49c2-8efd-9e19ca1e3ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.310275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e7314-1112-49c2-8efd-9e19ca1e3ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:03.310668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e7314-1112-49c2-8efd-9e19ca1e3ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.312126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e7314-1112-49c2-8efd-9e19ca1e3ec9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.330747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 368/748] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:03.331786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e7314-1112-49c2-8efd-9e19ca1e3ec9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:03.376880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd9bb2-c887-4f05-a045-4477712df7df None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:03.417516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58a292-7976-45e0-92a4-44af1b9ab180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:03.418320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58a292-7976-45e0-92a4-44af1b9ab180 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.418665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58a292-7976-45e0-92a4-44af1b9ab180 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.419001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58a292-7976-45e0-92a4-44af1b9ab180 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.419790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58a292-7976-45e0-92a4-44af1b9ab180 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:03.420158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58a292-7976-45e0-92a4-44af1b9ab180 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.421733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58a292-7976-45e0-92a4-44af1b9ab180 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.445919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 381/749] 10.222.0.100 () {56 vars in 1140 bytes} [Thu Sep 1 17:24:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:03.446946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58a292-7976-45e0-92a4-44af1b9ab180 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:03.461480 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e7314-1112-49c2-8efd-9e19ca1e3ec9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:03.509037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e94345-9612-44ea-808c-6a3a169dba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:03.509865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e94345-9612-44ea-808c-6a3a169dba25 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.510152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e94345-9612-44ea-808c-6a3a169dba25 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.510431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e94345-9612-44ea-808c-6a3a169dba25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.520285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e94345-9612-44ea-808c-6a3a169dba25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.523735 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e94345-9612-44ea-808c-6a3a169dba25 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-43e94345-9612-44ea-808c-6a3a169dba25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:03.539414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:03.540510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 369/750] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:03.542532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e94345-9612-44ea-808c-6a3a169dba25 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:03.550107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:03.558866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58a292-7976-45e0-92a4-44af1b9ab180 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:03.560084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:03.600374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce92ea-1ada-484e-acf3-7f7358fbef14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:03.601342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce92ea-1ada-484e-acf3-7f7358fbef14 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.601659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce92ea-1ada-484e-acf3-7f7358fbef14 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.602016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce92ea-1ada-484e-acf3-7f7358fbef14 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.611744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce92ea-1ada-484e-acf3-7f7358fbef14 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.613176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce92ea-1ada-484e-acf3-7f7358fbef14 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-fbce92ea-1ada-484e-acf3-7f7358fbef14 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:03.634365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:03.635396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:03.636336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:03.638761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 382/751] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:24:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:03.639781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce92ea-1ada-484e-acf3-7f7358fbef14 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:03.686924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e94345-9612-44ea-808c-6a3a169dba25 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:03.725254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bb2d2-2809-4f95-9d3c-455d2c4ee397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:03.725908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bb2d2-2809-4f95-9d3c-455d2c4ee397 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.726187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bb2d2-2809-4f95-9d3c-455d2c4ee397 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.726464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bb2d2-2809-4f95-9d3c-455d2c4ee397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.753966 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce92ea-1ada-484e-acf3-7f7358fbef14 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:03.791928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ed124-e8de-4ec7-ae5e-64ff0d796758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:03.792459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ed124-e8de-4ec7-ae5e-64ff0d796758 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.792750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ed124-e8de-4ec7-ae5e-64ff0d796758 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.793036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ed124-e8de-4ec7-ae5e-64ff0d796758 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.793744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ed124-e8de-4ec7-ae5e-64ff0d796758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:03.794075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ed124-e8de-4ec7-ae5e-64ff0d796758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.795505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ed124-e8de-4ec7-ae5e-64ff0d796758 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.801603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bb2d2-2809-4f95-9d3c-455d2c4ee397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.803792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bb2d2-2809-4f95-9d3c-455d2c4ee397 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.808466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 383/752] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:03.809325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ed124-e8de-4ec7-ae5e-64ff0d796758 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:03.820007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:03.830265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:03.917529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 370/753] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:03 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:24:03.919166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89b83b-fc58-4623-ada6-6eb3579f795c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.919528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89b83b-fc58-4623-ada6-6eb3579f795c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.919869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89b83b-fc58-4623-ada6-6eb3579f795c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.921292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 371/754] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:03.925815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89b83b-fc58-4623-ada6-6eb3579f795c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:03.930964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ed124-e8de-4ec7-ae5e-64ff0d796758 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:03.969865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1dea6-c8c6-4c4d-add0-b0d526b0ca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:03.970535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1dea6-c8c6-4c4d-add0-b0d526b0ca4b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:03.970822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1dea6-c8c6-4c4d-add0-b0d526b0ca4b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:03.971096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1dea6-c8c6-4c4d-add0-b0d526b0ca4b None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:03.971843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1dea6-c8c6-4c4d-add0-b0d526b0ca4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:03.972653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1dea6-c8c6-4c4d-add0-b0d526b0ca4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:03.974501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1dea6-c8c6-4c4d-add0-b0d526b0ca4b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:03.993962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 384/755] 10.222.0.100 () {56 vars in 1154 bytes} [Thu Sep 1 17:24:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:03.994807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1dea6-c8c6-4c4d-add0-b0d526b0ca4b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:04.049565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89b83b-fc58-4623-ada6-6eb3579f795c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:04.063511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:04.080883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:04.113225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b29c5f-375b-4eeb-99e7-5cd981d69217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:04.114343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b29c5f-375b-4eeb-99e7-5cd981d69217 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.114853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b29c5f-375b-4eeb-99e7-5cd981d69217 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.115907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b29c5f-375b-4eeb-99e7-5cd981d69217 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/c34f6f3e93b54a5d92b6f2f9f0de40f6/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.119476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b29c5f-375b-4eeb-99e7-5cd981d69217 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=c34f6f3e93b54a5d92b6f2f9f0de40f6, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:04.121673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b29c5f-375b-4eeb-99e7-5cd981d69217 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:04.140687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1dea6-c8c6-4c4d-add0-b0d526b0ca4b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:04.151416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 372/756] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:24:03 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/c34f6f3e93b54a5d92b6f2f9f0de40f6/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:24:04.151416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347c388-163e-495f-92dd-6cfee046ff65 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.151416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347c388-163e-495f-92dd-6cfee046ff65 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.151416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347c388-163e-495f-92dd-6cfee046ff65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.171331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2347c388-163e-495f-92dd-6cfee046ff65 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:04.172618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2347c388-163e-495f-92dd-6cfee046ff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:04.197629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0755904-710a-431a-9261-211af21e7c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:04.198405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0755904-710a-431a-9261-211af21e7c5c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.198976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0755904-710a-431a-9261-211af21e7c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.199256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0755904-710a-431a-9261-211af21e7c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.213788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0755904-710a-431a-9261-211af21e7c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:04.215472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0755904-710a-431a-9261-211af21e7c5c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b0755904-710a-431a-9261-211af21e7c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:04.236760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:04.238037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:04.239095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:04.239095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:04.239095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:04.241001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 385/757] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:04.241984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0755904-710a-431a-9261-211af21e7c5c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:04.263258 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 373/758] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:04 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:04.264702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347c388-163e-495f-92dd-6cfee046ff65 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:04.384577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0755904-710a-431a-9261-211af21e7c5c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:04.399434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:04.403297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347c388-163e-495f-92dd-6cfee046ff65 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:04.426724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2d750-1127-4ec3-a238-978b8acc6917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:04.427333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2d750-1127-4ec3-a238-978b8acc6917 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.427911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2d750-1127-4ec3-a238-978b8acc6917 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.428213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2d750-1127-4ec3-a238-978b8acc6917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.441986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:04.442610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf82111-2898-4f3f-8b8d-fc5931759db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:04.443223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf82111-2898-4f3f-8b8d-fc5931759db5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.443654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf82111-2898-4f3f-8b8d-fc5931759db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.443956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf82111-2898-4f3f-8b8d-fc5931759db5 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.444610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf82111-2898-4f3f-8b8d-fc5931759db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:04.444916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf82111-2898-4f3f-8b8d-fc5931759db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:04.446404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf82111-2898-4f3f-8b8d-fc5931759db5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:04.459934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 374/759] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:04.461360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9934c22-7d3b-4136-aa4f-162464fc8dd3 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.461671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9934c22-7d3b-4136-aa4f-162464fc8dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.461951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9934c22-7d3b-4136-aa4f-162464fc8dd3 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.463458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 375/760] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:04.468413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9934c22-7d3b-4136-aa4f-162464fc8dd3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:04.523800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2d750-1127-4ec3-a238-978b8acc6917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:04.526010 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2d750-1127-4ec3-a238-978b8acc6917 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:04.586247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9934c22-7d3b-4136-aa4f-162464fc8dd3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:04.627117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26343a6-cb95-4456-9091-e4e02f440fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:04.627751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26343a6-cb95-4456-9091-e4e02f440fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.628026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26343a6-cb95-4456-9091-e4e02f440fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.628294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26343a6-cb95-4456-9091-e4e02f440fc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.639091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26343a6-cb95-4456-9091-e4e02f440fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:04.640521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26343a6-cb95-4456-9091-e4e02f440fc4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:04.648042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 386/761] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:04 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:24:04.649907 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e304e-93be-42e6-8447-ae3bd4b2c402 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.650363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e304e-93be-42e6-8447-ae3bd4b2c402 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.650804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e304e-93be-42e6-8447-ae3bd4b2c402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c26343a6-cb95-4456-9091-e4e02f440fc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:04.653064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:04.653999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:04.654908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:04.654908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:04.654908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:04.654908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:04.654908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:04.654908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:04.654908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:04.654908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:04.654908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 376/762] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:04.655407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26343a6-cb95-4456-9091-e4e02f440fc4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:04.680704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d64e304e-93be-42e6-8447-ae3bd4b2c402 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:04.682101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d64e304e-93be-42e6-8447-ae3bd4b2c402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:04.684744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:04.702812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:04.771353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 387/763] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:04 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:04.773462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e304e-93be-42e6-8447-ae3bd4b2c402 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:04.781393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26343a6-cb95-4456-9091-e4e02f440fc4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:04.824113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390e71-f84e-4389-8ab3-e00a956470a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:04.825591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390e71-f84e-4389-8ab3-e00a956470a6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.825885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390e71-f84e-4389-8ab3-e00a956470a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.826220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390e71-f84e-4389-8ab3-e00a956470a6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.826817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03390e71-f84e-4389-8ab3-e00a956470a6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:04.828188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03390e71-f84e-4389-8ab3-e00a956470a6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-03390e71-f84e-4389-8ab3-e00a956470a6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:04.839878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:04.840938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:24:04.841808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:04.845717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 377/764] 10.222.0.100 () {56 vars in 1134 bytes} [Thu Sep 1 17:24:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:04.846748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390e71-f84e-4389-8ab3-e00a956470a6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:04.914088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e304e-93be-42e6-8447-ae3bd4b2c402 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:04.957781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a31c44-77ae-4de8-82f6-06b7930c48ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:04.958610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:04.960538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a31c44-77ae-4de8-82f6-06b7930c48ef None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:04.960829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a31c44-77ae-4de8-82f6-06b7930c48ef None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:04.961280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a31c44-77ae-4de8-82f6-06b7930c48ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:04.973277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a31c44-77ae-4de8-82f6-06b7930c48ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:04.975540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a31c44-77ae-4de8-82f6-06b7930c48ef None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:04.979451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390e71-f84e-4389-8ab3-e00a956470a6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-76a31c44-77ae-4de8-82f6-06b7930c48ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:04.992035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:04.993023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:04.993935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:04.993935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:04.993935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:04.998122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 388/765] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:04.998779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a31c44-77ae-4de8-82f6-06b7930c48ef None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.009510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:05.026855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62a828-cae4-469f-b4b0-916b51ebc310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.028055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62a828-cae4-469f-b4b0-916b51ebc310 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.028194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62a828-cae4-469f-b4b0-916b51ebc310 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.028358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62a828-cae4-469f-b4b0-916b51ebc310 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.029030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62a828-cae4-469f-b4b0-916b51ebc310 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:05.029334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62a828-cae4-469f-b4b0-916b51ebc310 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.030834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62a828-cae4-469f-b4b0-916b51ebc310 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.049429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 378/766] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:05.050378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62a828-cae4-469f-b4b0-916b51ebc310 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.117026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a31c44-77ae-4de8-82f6-06b7930c48ef None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.153931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982575a0-0723-4f5a-946f-1132250d636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.154537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982575a0-0723-4f5a-946f-1132250d636b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.154819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982575a0-0723-4f5a-946f-1132250d636b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.155092 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982575a0-0723-4f5a-946f-1132250d636b None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.155724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982575a0-0723-4f5a-946f-1132250d636b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:05.156027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982575a0-0723-4f5a-946f-1132250d636b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.157382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982575a0-0723-4f5a-946f-1132250d636b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.159167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62a828-cae4-469f-b4b0-916b51ebc310 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.163568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 389/767] 10.222.0.100 () {56 vars in 1129 bytes} [Thu Sep 1 17:24:04 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:05.164655 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982575a0-0723-4f5a-946f-1132250d636b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.197207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b31947-aac6-444c-b44a-77480791b0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.197840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b31947-aac6-444c-b44a-77480791b0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.198179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b31947-aac6-444c-b44a-77480791b0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.198392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b31947-aac6-444c-b44a-77480791b0f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.199035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b31947-aac6-444c-b44a-77480791b0f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:05.199345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b31947-aac6-444c-b44a-77480791b0f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.200760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b31947-aac6-444c-b44a-77480791b0f5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.218009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 379/768] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:05.218009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b31947-aac6-444c-b44a-77480791b0f5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.220503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:05.248845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:05.284463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982575a0-0723-4f5a-946f-1132250d636b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.321207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c8e0-447a-40c3-a637-e676bc4af93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.322129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c8e0-447a-40c3-a637-e676bc4af93f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.322414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c8e0-447a-40c3-a637-e676bc4af93f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.322692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c8e0-447a-40c3-a637-e676bc4af93f None admin] PATH_INFO: `/v3/services/network` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.323277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88c8e0-447a-40c3-a637-e676bc4af93f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.324638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88c8e0-447a-40c3-a637-e676bc4af93f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.325881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b31947-aac6-444c-b44a-77480791b0f5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4b88c8e0-447a-40c3-a637-e676bc4af93f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:05.334919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:05.341106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:05.342202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 390/769] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:05.343216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c8e0-447a-40c3-a637-e676bc4af93f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.372842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347253b3-c6a2-4904-81e7-7f9fbad59c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.373611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347253b3-c6a2-4904-81e7-7f9fbad59c76 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.373964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347253b3-c6a2-4904-81e7-7f9fbad59c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.374425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347253b3-c6a2-4904-81e7-7f9fbad59c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.375097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347253b3-c6a2-4904-81e7-7f9fbad59c76 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=f3fa2499f5594b7798f6fe4443a14f4c) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:05.375473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347253b3-c6a2-4904-81e7-7f9fbad59c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.377003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347253b3-c6a2-4904-81e7-7f9fbad59c76 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.382720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 380/770] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/role_assignments?user.id=f3fa2499f5594b7798f6fe4443a14f4c&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:05.383465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347253b3-c6a2-4904-81e7-7f9fbad59c76 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.478247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:05.479376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c8e0-447a-40c3-a637-e676bc4af93f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.520918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd10f-c0dc-454a-af11-f465fd5ac2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.521863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd10f-c0dc-454a-af11-f465fd5ac2be None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.522218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd10f-c0dc-454a-af11-f465fd5ac2be None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.522911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd10f-c0dc-454a-af11-f465fd5ac2be None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.523353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fd10f-c0dc-454a-af11-f465fd5ac2be None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:05.523726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fd10f-c0dc-454a-af11-f465fd5ac2be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.525279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fd10f-c0dc-454a-af11-f465fd5ac2be None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.526448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347253b3-c6a2-4904-81e7-7f9fbad59c76 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.527240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:05.534059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 391/771] 10.222.0.100 () {56 vars in 1129 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:05.534921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd10f-c0dc-454a-af11-f465fd5ac2be None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.575997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3c906-4575-4ddd-878c-23ac012631dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.577101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3c906-4575-4ddd-878c-23ac012631dd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.578460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3c906-4575-4ddd-878c-23ac012631dd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.578930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3c906-4575-4ddd-878c-23ac012631dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.587294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3c906-4575-4ddd-878c-23ac012631dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.590043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3c906-4575-4ddd-878c-23ac012631dd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8ca3c906-4575-4ddd-878c-23ac012631dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:05.605096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:05.606257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:05.607177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:05.607177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:05.607177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:05.607177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:05.607177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:05.607177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:05.607177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:05.607177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:05.607177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 381/772] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:05.607669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3c906-4575-4ddd-878c-23ac012631dd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.677681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd10f-c0dc-454a-af11-f465fd5ac2be None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.717838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b21e2-dff3-4bc7-8b91-a443336eccc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.718872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b21e2-dff3-4bc7-8b91-a443336eccc6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.719370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b21e2-dff3-4bc7-8b91-a443336eccc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.719840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b21e2-dff3-4bc7-8b91-a443336eccc6 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.720804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b21e2-dff3-4bc7-8b91-a443336eccc6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:05.721328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b21e2-dff3-4bc7-8b91-a443336eccc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.723032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b21e2-dff3-4bc7-8b91-a443336eccc6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.735053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3c906-4575-4ddd-878c-23ac012631dd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.735388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 392/773] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/services?name=network => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:05.736649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b21e2-dff3-4bc7-8b91-a443336eccc6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.764377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:05.774375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e36cf-a616-4298-bdfc-1b9b0729521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.774996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e36cf-a616-4298-bdfc-1b9b0729521d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.775275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e36cf-a616-4298-bdfc-1b9b0729521d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.775551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e36cf-a616-4298-bdfc-1b9b0729521d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.776305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e36cf-a616-4298-bdfc-1b9b0729521d None admin] RBAC: Adding query filter params (interface=public, service_id=b79af9f503f84f58bd45709279c63935, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:05.776568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e36cf-a616-4298-bdfc-1b9b0729521d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.778234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e36cf-a616-4298-bdfc-1b9b0729521d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.791402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 382/774] 10.222.0.100 () {56 vars in 1256 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/endpoints?service_id=b79af9f503f84f58bd45709279c63935&interface=public®ion_id=RegionOne => generated 195 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:05.792455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e36cf-a616-4298-bdfc-1b9b0729521d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.817224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:05.868421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b21e2-dff3-4bc7-8b91-a443336eccc6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.901372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e36cf-a616-4298-bdfc-1b9b0729521d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:05.905252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cae0d-4116-4776-a6a0-172be2facc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.906130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cae0d-4116-4776-a6a0-172be2facc91 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.906613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cae0d-4116-4776-a6a0-172be2facc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.907097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cae0d-4116-4776-a6a0-172be2facc91 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.908004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cae0d-4116-4776-a6a0-172be2facc91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:05.908521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cae0d-4116-4776-a6a0-172be2facc91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.910207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cae0d-4116-4776-a6a0-172be2facc91 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.934158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 393/775] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:05.936157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cae0d-4116-4776-a6a0-172be2facc91 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:05.940873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef58c03-0420-4564-bd37-cd67c668683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:05.941585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef58c03-0420-4564-bd37-cd67c668683e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:05.941959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef58c03-0420-4564-bd37-cd67c668683e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:05.942277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef58c03-0420-4564-bd37-cd67c668683e None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:05.942991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef58c03-0420-4564-bd37-cd67c668683e None admin] RBAC: Adding query filter params (type=network) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:05.944497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef58c03-0420-4564-bd37-cd67c668683e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:05.945099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef58c03-0420-4564-bd37-cd67c668683e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:05.958771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 383/776] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/services?type=network => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:05.958771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef58c03-0420-4564-bd37-cd67c668683e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:06.019801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:06.054591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cae0d-4116-4776-a6a0-172be2facc91 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:06.068766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef58c03-0420-4564-bd37-cd67c668683e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:06.080080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:06.094716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e097d1-6399-48ae-a26d-6d74aca7d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:06.095942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e097d1-6399-48ae-a26d-6d74aca7d5da None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:06.096369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e097d1-6399-48ae-a26d-6d74aca7d5da None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:06.096933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e097d1-6399-48ae-a26d-6d74aca7d5da None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:06.098050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e097d1-6399-48ae-a26d-6d74aca7d5da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:06.100743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e097d1-6399-48ae-a26d-6d74aca7d5da None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:06.106368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de7f71-4d93-4be2-8d6a-aee127091a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:06.107092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de7f71-4d93-4be2-8d6a-aee127091a6b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:06.107467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de7f71-4d93-4be2-8d6a-aee127091a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:06.107834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de7f71-4d93-4be2-8d6a-aee127091a6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:06.108616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 394/777] 10.222.0.100 () {56 vars in 1092 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/services => generated 776 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:06.108731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de7f71-4d93-4be2-8d6a-aee127091a6b None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=71b9335f3f744a31a84ab278d9b45e36) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:06.109673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e097d1-6399-48ae-a26d-6d74aca7d5da None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:06.114228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de7f71-4d93-4be2-8d6a-aee127091a6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:06.115896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de7f71-4d93-4be2-8d6a-aee127091a6b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:06.131977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 384/778] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:05 2022] GET /identity/v3/role_assignments?user.id=71b9335f3f744a31a84ab278d9b45e36&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:06.215194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8611c-d06b-4d42-952f-74c9cfa24640 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:06.215558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8611c-d06b-4d42-952f-74c9cfa24640 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:06.215772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8611c-d06b-4d42-952f-74c9cfa24640 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:06.217390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 385/779] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:06.227182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994e82c-702f-4d84-aee6-3e5a5e45e5de None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:06.227504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994e82c-702f-4d84-aee6-3e5a5e45e5de None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:06.227742 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994e82c-702f-4d84-aee6-3e5a5e45e5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:06.283729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e097d1-6399-48ae-a26d-6d74aca7d5da None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:06.286836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b994e82c-702f-4d84-aee6-3e5a5e45e5de None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:06.288755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b994e82c-702f-4d84-aee6-3e5a5e45e5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:06.332431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8a64-6431-4b09-96ee-083a9d917593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:06.333060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8a64-6431-4b09-96ee-083a9d917593 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:06.333334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8a64-6431-4b09-96ee-083a9d917593 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:06.333637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8a64-6431-4b09-96ee-083a9d917593 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:06.335121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc8a64-6431-4b09-96ee-083a9d917593 None admin] RBAC: Adding query filter params (interface=public, service_id=f57014adb13843fabc0d3926ec74f7d4, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:06.335434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc8a64-6431-4b09-96ee-083a9d917593 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:06.336864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc8a64-6431-4b09-96ee-083a9d917593 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:06.345473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 395/780] 10.222.0.100 () {56 vars in 1256 bytes} [Thu Sep 1 17:24:06 2022] GET /identity/v3/endpoints?service_id=f57014adb13843fabc0d3926ec74f7d4&interface=public®ion_id=RegionOne => generated 195 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:06.351023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8a64-6431-4b09-96ee-083a9d917593 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:06.400948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 386/781] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:06 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:06.457960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:06.475901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8a64-6431-4b09-96ee-083a9d917593 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:06.525404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca2b3-f4fe-4053-bd56-5df4cefc755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:06.526363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca2b3-f4fe-4053-bd56-5df4cefc755f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:06.526790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca2b3-f4fe-4053-bd56-5df4cefc755f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:06.527203 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca2b3-f4fe-4053-bd56-5df4cefc755f None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:06.528245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca2b3-f4fe-4053-bd56-5df4cefc755f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:06.529845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca2b3-f4fe-4053-bd56-5df4cefc755f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:06.536462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 396/782] 10.222.0.100 () {56 vars in 1092 bytes} [Thu Sep 1 17:24:06 2022] GET /identity/v3/services => generated 776 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:06.875457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994e82c-702f-4d84-aee6-3e5a5e45e5de None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:06.929459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:06.987538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994e82c-702f-4d84-aee6-3e5a5e45e5de None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:07.030180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f27d8-4431-4462-b692-410b9d53aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:07.030813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f27d8-4431-4462-b692-410b9d53aa4d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:07.031090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f27d8-4431-4462-b692-410b9d53aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:07.031405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f27d8-4431-4462-b692-410b9d53aa4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:07.050851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f27d8-4431-4462-b692-410b9d53aa4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-de7f27d8-4431-4462-b692-410b9d53aa4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:07.056094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:07.057038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:07.058284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:07.060680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 387/783] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:07.066294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca2b3-f4fe-4053-bd56-5df4cefc755f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:07.209932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca2b3-f4fe-4053-bd56-5df4cefc755f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:07.223613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:07.251354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c9f9e-e368-4f68-9ea1-05eea718ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:07.251962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c9f9e-e368-4f68-9ea1-05eea718ec9d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:07.252241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c9f9e-e368-4f68-9ea1-05eea718ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:07.252515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c9f9e-e368-4f68-9ea1-05eea718ec9d None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:07.253181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c9f9e-e368-4f68-9ea1-05eea718ec9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:07.253482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c9f9e-e368-4f68-9ea1-05eea718ec9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:07.254876 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c9f9e-e368-4f68-9ea1-05eea718ec9d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:07.267150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 397/784] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:24:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:07.274612 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f27d8-4431-4462-b692-410b9d53aa4d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:07.419292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f27d8-4431-4462-b692-410b9d53aa4d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:07.431855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:07.456151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b165a9-c588-4bd5-a83b-07a9226de8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:07.456841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b165a9-c588-4bd5-a83b-07a9226de8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:07.457095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b165a9-c588-4bd5-a83b-07a9226de8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:07.457417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b165a9-c588-4bd5-a83b-07a9226de8b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:07.468115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b165a9-c588-4bd5-a83b-07a9226de8b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:07.469495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b165a9-c588-4bd5-a83b-07a9226de8b4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-20b165a9-c588-4bd5-a83b-07a9226de8b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:07.484948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:07.486016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:07.486915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:07.486915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:07.486915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:07.486915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 388/785] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:07.493990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c9f9e-e368-4f68-9ea1-05eea718ec9d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:07.642229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c9f9e-e368-4f68-9ea1-05eea718ec9d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:07.762991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710afc77-81bf-42e8-a9be-419d21b80d6a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:07.763618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710afc77-81bf-42e8-a9be-419d21b80d6a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:07.764133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710afc77-81bf-42e8-a9be-419d21b80d6a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:07.766057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 389/786] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:07.772887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3a0d-7e60-4ccf-bf3d-eabbb8c26f5a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:07.773566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3a0d-7e60-4ccf-bf3d-eabbb8c26f5a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:07.774088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3a0d-7e60-4ccf-bf3d-eabbb8c26f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:07.793920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-be6a3a0d-7e60-4ccf-bf3d-eabbb8c26f5a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:07.795452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-be6a3a0d-7e60-4ccf-bf3d-eabbb8c26f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:07.826591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4bb88-734c-4d72-a67a-efe6582edb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:07.827452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4bb88-734c-4d72-a67a-efe6582edb71 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:07.827736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4bb88-734c-4d72-a67a-efe6582edb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:07.828010 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4bb88-734c-4d72-a67a-efe6582edb71 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:07.828668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4bb88-734c-4d72-a67a-efe6582edb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:07.828960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4bb88-734c-4d72-a67a-efe6582edb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:07.830893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4bb88-734c-4d72-a67a-efe6582edb71 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:07.861908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 398/787] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:07.875233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4bb88-734c-4d72-a67a-efe6582edb71 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:07.916138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 390/788] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:07 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:07.935525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:08.012073 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4bb88-734c-4d72-a67a-efe6582edb71 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:08.049461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bd44a-26e5-40a7-a27b-b44375c1df4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:08.050496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bd44a-26e5-40a7-a27b-b44375c1df4f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.050987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bd44a-26e5-40a7-a27b-b44375c1df4f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.051451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bd44a-26e5-40a7-a27b-b44375c1df4f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.059616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bd44a-26e5-40a7-a27b-b44375c1df4f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:08.061311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bd44a-26e5-40a7-a27b-b44375c1df4f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-363bd44a-26e5-40a7-a27b-b44375c1df4f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:08.074801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:08.076213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:08.077252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:08.079764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 399/789] 10.222.0.100 () {56 vars in 1098 bytes} [Thu Sep 1 17:24:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:08.086455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a3a0d-7e60-4ccf-bf3d-eabbb8c26f5a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:08.191103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:08.200961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a3a0d-7e60-4ccf-bf3d-eabbb8c26f5a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:08.294469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bd44a-26e5-40a7-a27b-b44375c1df4f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:08.418305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a2621-3862-436e-9005-205f659e02d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:08.418937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a2621-3862-436e-9005-205f659e02d5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.419221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a2621-3862-436e-9005-205f659e02d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.419492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a2621-3862-436e-9005-205f659e02d5 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.420238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a2621-3862-436e-9005-205f659e02d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:08.420542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a2621-3862-436e-9005-205f659e02d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:08.422030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a2621-3862-436e-9005-205f659e02d5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:08.433971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:08.441520 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bd44a-26e5-40a7-a27b-b44375c1df4f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:08.463865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 391/790] 10.222.0.100 () {56 vars in 1138 bytes} [Thu Sep 1 17:24:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:08.470581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a2621-3862-436e-9005-205f659e02d5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:08.482445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf143ae-c219-4151-bd18-1a79bcec7046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:08.483469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf143ae-c219-4151-bd18-1a79bcec7046 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.484101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf143ae-c219-4151-bd18-1a79bcec7046 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.484581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf143ae-c219-4151-bd18-1a79bcec7046 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.497229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf143ae-c219-4151-bd18-1a79bcec7046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-aaf143ae-c219-4151-bd18-1a79bcec7046 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:08.503383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:08.504332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:08.505170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:08.508723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 400/791] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:08.514976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf143ae-c219-4151-bd18-1a79bcec7046 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:08.645830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a2621-3862-436e-9005-205f659e02d5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:08.649797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf143ae-c219-4151-bd18-1a79bcec7046 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:08.684906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48dc7-4385-4554-a055-12943fe5f515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:08.685412 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48dc7-4385-4554-a055-12943fe5f515 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.685801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48dc7-4385-4554-a055-12943fe5f515 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.686079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48dc7-4385-4554-a055-12943fe5f515 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.691702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071afe6-d585-4841-9e76-a61a8af51653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:08.692387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071afe6-d585-4841-9e76-a61a8af51653 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.692679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071afe6-d585-4841-9e76-a61a8af51653 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.692956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071afe6-d585-4841-9e76-a61a8af51653 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.694185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071afe6-d585-4841-9e76-a61a8af51653 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:08.694493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071afe6-d585-4841-9e76-a61a8af51653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:08.695873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071afe6-d585-4841-9e76-a61a8af51653 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:08.705298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 401/792] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:24:08 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:08.706208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e74a5-d693-4a43-b384-bb5376ff5ff6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.706496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e74a5-d693-4a43-b384-bb5376ff5ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.706773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e74a5-d693-4a43-b384-bb5376ff5ff6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.708110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 402/793] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:08 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:08.709663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e48d0a-e4c3-4bb0-957a-360f198a3ed0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.709909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e48d0a-e4c3-4bb0-957a-360f198a3ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.710186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e48d0a-e4c3-4bb0-957a-360f198a3ed0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.711085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:08.713744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 403/794] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:08.713744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e48d0a-e4c3-4bb0-957a-360f198a3ed0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:08.734767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:08.768282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48dc7-4385-4554-a055-12943fe5f515 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:08.769886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48dc7-4385-4554-a055-12943fe5f515 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:08.837247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e48d0a-e4c3-4bb0-957a-360f198a3ed0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:08.873418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 392/795] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:08 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:24:08.874669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde297d-bf3f-467c-b54e-bf05308e7892 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.874959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde297d-bf3f-467c-b54e-bf05308e7892 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.875231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde297d-bf3f-467c-b54e-bf05308e7892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.879264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c478076-f1c5-4976-9638-554ebe5f0e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:08.879902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c478076-f1c5-4976-9638-554ebe5f0e9e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.880177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c478076-f1c5-4976-9638-554ebe5f0e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.880441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c478076-f1c5-4976-9638-554ebe5f0e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.892881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c478076-f1c5-4976-9638-554ebe5f0e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:08.893414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4bde297d-bf3f-467c-b54e-bf05308e7892 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:08.894404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c478076-f1c5-4976-9638-554ebe5f0e9e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:08.894623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4bde297d-bf3f-467c-b54e-bf05308e7892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0c478076-f1c5-4976-9638-554ebe5f0e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:08.911660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:08.912727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:08.913653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:08.913653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:08.913653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:08.913653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 404/796] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:08.914483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3021c99-7d1f-4cd8-bb29-5d7bac20fa0b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:08.914764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3021c99-7d1f-4cd8-bb29-5d7bac20fa0b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:08.915032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3021c99-7d1f-4cd8-bb29-5d7bac20fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:08.934294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a3021c99-7d1f-4cd8-bb29-5d7bac20fa0b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:08.935539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a3021c99-7d1f-4cd8-bb29-5d7bac20fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:08.948888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:08.981598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 393/797] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:08 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:08.981598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde297d-bf3f-467c-b54e-bf05308e7892 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.021907 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:09.028913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 405/798] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:08 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:09.029897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3021c99-7d1f-4cd8-bb29-5d7bac20fa0b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.117675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde297d-bf3f-467c-b54e-bf05308e7892 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.157320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1853db-092b-4633-aea5-439587bd3247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:09.157979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1853db-092b-4633-aea5-439587bd3247 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.158263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1853db-092b-4633-aea5-439587bd3247 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.158538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1853db-092b-4633-aea5-439587bd3247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.158664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3021c99-7d1f-4cd8-bb29-5d7bac20fa0b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.175372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1853db-092b-4633-aea5-439587bd3247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:09.177407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1853db-092b-4633-aea5-439587bd3247 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1b1853db-092b-4633-aea5-439587bd3247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:09.197465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:09.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:09.199576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:09.199576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:09.199576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:09.199576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 394/799] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:09.200232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dcc31-6aee-4ea4-b4ff-2b11e0c9c141 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.200232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dcc31-6aee-4ea4-b4ff-2b11e0c9c141 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.200524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dcc31-6aee-4ea4-b4ff-2b11e0c9c141 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.205733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 395/800] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:09 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:09.206577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dcc31-6aee-4ea4-b4ff-2b11e0c9c141 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.208162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736564c-0749-4b86-9761-a00bec443ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:09.208879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736564c-0749-4b86-9761-a00bec443ffb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.209790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736564c-0749-4b86-9761-a00bec443ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.210417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736564c-0749-4b86-9761-a00bec443ffb None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.211110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736564c-0749-4b86-9761-a00bec443ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:09.212495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736564c-0749-4b86-9761-a00bec443ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:09.214090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736564c-0749-4b86-9761-a00bec443ffb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:09.232640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 406/801] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:09.234194 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14628e-64a9-42c2-9774-fbc503af5d15 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.234499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14628e-64a9-42c2-9774-fbc503af5d15 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.235382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14628e-64a9-42c2-9774-fbc503af5d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.244847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:09.260440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bc14628e-64a9-42c2-9774-fbc503af5d15 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:09.262156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bc14628e-64a9-42c2-9774-fbc503af5d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:09.326038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dcc31-6aee-4ea4-b4ff-2b11e0c9c141 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.329720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:09.349600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 407/802] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:09 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:09.349600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14628e-64a9-42c2-9774-fbc503af5d15 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.361727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfc950-c902-42b5-8cb9-a58c7ebf5c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:09.362339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfc950-c902-42b5-8cb9-a58c7ebf5c43 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.362617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfc950-c902-42b5-8cb9-a58c7ebf5c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.362887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfc950-c902-42b5-8cb9-a58c7ebf5c43 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.363525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfc950-c902-42b5-8cb9-a58c7ebf5c43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:09.363825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfc950-c902-42b5-8cb9-a58c7ebf5c43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:09.365365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfc950-c902-42b5-8cb9-a58c7ebf5c43 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:09.381981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 396/803] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:09.381981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfc950-c902-42b5-8cb9-a58c7ebf5c43 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.463045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14628e-64a9-42c2-9774-fbc503af5d15 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.479782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:09.489830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfc950-c902-42b5-8cb9-a58c7ebf5c43 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.498689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa0caf-64ad-4340-89b0-218db93d2836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:09.499525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa0caf-64ad-4340-89b0-218db93d2836 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.499640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa0caf-64ad-4340-89b0-218db93d2836 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.499855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa0caf-64ad-4340-89b0-218db93d2836 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.507651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa0caf-64ad-4340-89b0-218db93d2836 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:09.508973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa0caf-64ad-4340-89b0-218db93d2836 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-05fa0caf-64ad-4340-89b0-218db93d2836 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:09.520730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:09.522305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 408/804] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:09.523276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa0caf-64ad-4340-89b0-218db93d2836 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.531711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230588b2-6665-42d6-9f70-cc7cc2c341b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:09.532338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230588b2-6665-42d6-9f70-cc7cc2c341b5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.532612 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230588b2-6665-42d6-9f70-cc7cc2c341b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.532885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230588b2-6665-42d6-9f70-cc7cc2c341b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.544351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230588b2-6665-42d6-9f70-cc7cc2c341b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-230588b2-6665-42d6-9f70-cc7cc2c341b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:09.549173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:09.550141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:09.550997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 397/805] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:09.551687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230588b2-6665-42d6-9f70-cc7cc2c341b5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.588537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:09.641543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa0caf-64ad-4340-89b0-218db93d2836 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.669217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230588b2-6665-42d6-9f70-cc7cc2c341b5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.682978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32b9c4-02c3-424c-9330-b866dad9f75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:09.683609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32b9c4-02c3-424c-9330-b866dad9f75a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.683888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32b9c4-02c3-424c-9330-b866dad9f75a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.684157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32b9c4-02c3-424c-9330-b866dad9f75a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.684743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32b9c4-02c3-424c-9330-b866dad9f75a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:09.686144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32b9c4-02c3-424c-9330-b866dad9f75a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6a32b9c4-02c3-424c-9330-b866dad9f75a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:09.694793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:09.696057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:09.697147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 409/806] 10.222.0.100 () {56 vars in 1134 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:09.698349 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32b9c4-02c3-424c-9330-b866dad9f75a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.706242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb95666-5f1a-4168-a001-ee6e962c4f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:09.706871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb95666-5f1a-4168-a001-ee6e962c4f14 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.707151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb95666-5f1a-4168-a001-ee6e962c4f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.707418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb95666-5f1a-4168-a001-ee6e962c4f14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.714796 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb95666-5f1a-4168-a001-ee6e962c4f14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:09.716150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb95666-5f1a-4168-a001-ee6e962c4f14 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-bdb95666-5f1a-4168-a001-ee6e962c4f14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:09.727663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:09.728752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 398/807] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:09.729894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb95666-5f1a-4168-a001-ee6e962c4f14 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.744937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:09.802773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32b9c4-02c3-424c-9330-b866dad9f75a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.816845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:09.835769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb95666-5f1a-4168-a001-ee6e962c4f14 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.840902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede19658-28b9-48e7-8d90-2fc8f0a830b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:09.841470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede19658-28b9-48e7-8d90-2fc8f0a830b3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.841787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede19658-28b9-48e7-8d90-2fc8f0a830b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.842060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede19658-28b9-48e7-8d90-2fc8f0a830b3 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.842791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede19658-28b9-48e7-8d90-2fc8f0a830b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:09.843097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede19658-28b9-48e7-8d90-2fc8f0a830b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:09.844515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede19658-28b9-48e7-8d90-2fc8f0a830b3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:09.867176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 410/808] 10.222.0.100 () {56 vars in 1140 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:09.867176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede19658-28b9-48e7-8d90-2fc8f0a830b3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.870861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45d266-f46d-4036-ab6b-299af15bc1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:09.871464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45d266-f46d-4036-ab6b-299af15bc1db None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:09.871743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45d266-f46d-4036-ab6b-299af15bc1db None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:09.872011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45d266-f46d-4036-ab6b-299af15bc1db None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:09.872684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45d266-f46d-4036-ab6b-299af15bc1db None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:09.872982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45d266-f46d-4036-ab6b-299af15bc1db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:09.874372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45d266-f46d-4036-ab6b-299af15bc1db None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:09.882642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 399/809] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:09.883507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45d266-f46d-4036-ab6b-299af15bc1db None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:09.961682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:09.971026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede19658-28b9-48e7-8d90-2fc8f0a830b3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:09.987192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45d266-f46d-4036-ab6b-299af15bc1db None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.006816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a772e5b-b95d-40ec-897c-953ac99d454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.007436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a772e5b-b95d-40ec-897c-953ac99d454b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.007715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a772e5b-b95d-40ec-897c-953ac99d454b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.007984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a772e5b-b95d-40ec-897c-953ac99d454b None admin] PATH_INFO: `/v3/services/network` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.008564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a772e5b-b95d-40ec-897c-953ac99d454b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.009956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a772e5b-b95d-40ec-897c-953ac99d454b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5a772e5b-b95d-40ec-897c-953ac99d454b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:10.018601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:10.019806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:10.020843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 411/810] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:10.022903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a772e5b-b95d-40ec-897c-953ac99d454b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.026516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8c3fd-4b56-4388-b1d3-569e5e802785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.027133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8c3fd-4b56-4388-b1d3-569e5e802785 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.027401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8c3fd-4b56-4388-b1d3-569e5e802785 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.027929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8c3fd-4b56-4388-b1d3-569e5e802785 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.028571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8c3fd-4b56-4388-b1d3-569e5e802785 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:10.028867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8c3fd-4b56-4388-b1d3-569e5e802785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.030239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8c3fd-4b56-4388-b1d3-569e5e802785 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.034807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 400/811] 10.222.0.100 () {56 vars in 1129 bytes} [Thu Sep 1 17:24:09 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:10.035572 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8c3fd-4b56-4388-b1d3-569e5e802785 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.058280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:10.122717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a772e5b-b95d-40ec-897c-953ac99d454b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.137456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8c3fd-4b56-4388-b1d3-569e5e802785 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.156928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803db907-41cf-4b3d-9a03-1b756d2f7602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.157530 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803db907-41cf-4b3d-9a03-1b756d2f7602 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.157840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803db907-41cf-4b3d-9a03-1b756d2f7602 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.158113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803db907-41cf-4b3d-9a03-1b756d2f7602 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.158801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803db907-41cf-4b3d-9a03-1b756d2f7602 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:10.159337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803db907-41cf-4b3d-9a03-1b756d2f7602 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.160521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803db907-41cf-4b3d-9a03-1b756d2f7602 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.173085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26938685-8218-4cb0-9f4d-7c78f304c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.173744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26938685-8218-4cb0-9f4d-7c78f304c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.174022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26938685-8218-4cb0-9f4d-7c78f304c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.174254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 412/812] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:10.174407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26938685-8218-4cb0-9f4d-7c78f304c8e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.175171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803db907-41cf-4b3d-9a03-1b756d2f7602 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.182867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:10.186876 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26938685-8218-4cb0-9f4d-7c78f304c8e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.188567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26938685-8218-4cb0-9f4d-7c78f304c8e6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-26938685-8218-4cb0-9f4d-7c78f304c8e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:10.208107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:10.209668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:10.210591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:10.210591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:10.210591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:10.210591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 401/813] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:10.210591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26938685-8218-4cb0-9f4d-7c78f304c8e6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.280278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:10.286494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803db907-41cf-4b3d-9a03-1b756d2f7602 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.310205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26938685-8218-4cb0-9f4d-7c78f304c8e6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.321600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c178f4-4970-46ed-a114-811e57bb1c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.322239 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c178f4-4970-46ed-a114-811e57bb1c1d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.322519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c178f4-4970-46ed-a114-811e57bb1c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.322790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c178f4-4970-46ed-a114-811e57bb1c1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.333273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c178f4-4970-46ed-a114-811e57bb1c1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.335298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c178f4-4970-46ed-a114-811e57bb1c1d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.345109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deaa7a0-64a2-48d6-8bea-6acb6b069da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.345735 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaa7a0-64a2-48d6-8bea-6acb6b069da8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.346014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaa7a0-64a2-48d6-8bea-6acb6b069da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.346284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaa7a0-64a2-48d6-8bea-6acb6b069da8 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.346896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deaa7a0-64a2-48d6-8bea-6acb6b069da8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:10.347191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deaa7a0-64a2-48d6-8bea-6acb6b069da8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.348537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deaa7a0-64a2-48d6-8bea-6acb6b069da8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c9c178f4-4970-46ed-a114-811e57bb1c1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:10.350761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:10.351677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:10.352570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:10.352570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:10.352570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:10.352570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 413/814] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:10.352802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c178f4-4970-46ed-a114-811e57bb1c1d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.353221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 402/815] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/services?name=network => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:10.354094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deaa7a0-64a2-48d6-8bea-6acb6b069da8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.409811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:10.455605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deaa7a0-64a2-48d6-8bea-6acb6b069da8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.456986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c178f4-4970-46ed-a114-811e57bb1c1d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.488281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2f361-a1d7-4f84-8165-e839048f744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.488871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2f361-a1d7-4f84-8165-e839048f744d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.489145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2f361-a1d7-4f84-8165-e839048f744d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.489410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2f361-a1d7-4f84-8165-e839048f744d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.490671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2f361-a1d7-4f84-8165-e839048f744d None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:10.491125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2f361-a1d7-4f84-8165-e839048f744d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.491290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa959d4-2645-493c-8407-84c1b6f6a5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.491749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa959d4-2645-493c-8407-84c1b6f6a5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.492041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa959d4-2645-493c-8407-84c1b6f6a5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.492300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa959d4-2645-493c-8407-84c1b6f6a5b7 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.492707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2f361-a1d7-4f84-8165-e839048f744d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.492923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa959d4-2645-493c-8407-84c1b6f6a5b7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:10.493225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa959d4-2645-493c-8407-84c1b6f6a5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.494601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa959d4-2645-493c-8407-84c1b6f6a5b7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.498621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 403/816] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/role_assignments?user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 610 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:10.504041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2f361-a1d7-4f84-8165-e839048f744d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.504257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 414/817] 10.222.0.100 () {56 vars in 1129 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:10.504257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa959d4-2645-493c-8407-84c1b6f6a5b7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.517490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:10.624065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa959d4-2645-493c-8407-84c1b6f6a5b7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.631696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2f361-a1d7-4f84-8165-e839048f744d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.652201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:10.673088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154c30-0061-4bfa-b474-30158916632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.673088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154c30-0061-4bfa-b474-30158916632b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.673088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154c30-0061-4bfa-b474-30158916632b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.673458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aff3f-c3fa-41f3-a052-e3ed7cc7407d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.673555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154c30-0061-4bfa-b474-30158916632b None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.673860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aff3f-c3fa-41f3-a052-e3ed7cc7407d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.674032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aff3f-c3fa-41f3-a052-e3ed7cc7407d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.674129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9154c30-0061-4bfa-b474-30158916632b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:10.675018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aff3f-c3fa-41f3-a052-e3ed7cc7407d None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.675127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9154c30-0061-4bfa-b474-30158916632b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.675201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aff3f-c3fa-41f3-a052-e3ed7cc7407d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:10.675282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aff3f-c3fa-41f3-a052-e3ed7cc7407d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.676825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9154c30-0061-4bfa-b474-30158916632b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.676930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aff3f-c3fa-41f3-a052-e3ed7cc7407d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.690570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 415/818] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:10.690570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154c30-0061-4bfa-b474-30158916632b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.690915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 404/819] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:10.691696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aff3f-c3fa-41f3-a052-e3ed7cc7407d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.783766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:10.801981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aff3f-c3fa-41f3-a052-e3ed7cc7407d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.819538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154c30-0061-4bfa-b474-30158916632b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:10.838927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545227ba-ec9a-4683-9e0d-736b5365e69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.839535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545227ba-ec9a-4683-9e0d-736b5365e69e None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.839814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545227ba-ec9a-4683-9e0d-736b5365e69e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.840083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545227ba-ec9a-4683-9e0d-736b5365e69e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.840755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545227ba-ec9a-4683-9e0d-736b5365e69e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.842154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545227ba-ec9a-4683-9e0d-736b5365e69e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.859673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789195-e123-4886-a9bc-1db0cfd8e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:10.860447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789195-e123-4886-a9bc-1db0cfd8e633 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:10.860726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789195-e123-4886-a9bc-1db0cfd8e633 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:10.860997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789195-e123-4886-a9bc-1db0cfd8e633 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:10.861651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d789195-e123-4886-a9bc-1db0cfd8e633 None admin] RBAC: Adding query filter params (type=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:10.861961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d789195-e123-4886-a9bc-1db0cfd8e633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:10.863437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d789195-e123-4886-a9bc-1db0cfd8e633 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:10.865302 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 405/820] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:24:10 2022] POST /identity/v3/endpoints => generated 348 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:10.866139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545227ba-ec9a-4683-9e0d-736b5365e69e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.889038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 416/821] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/services?type=network => generated 352 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:10.890810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789195-e123-4886-a9bc-1db0cfd8e633 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:10.911359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:11.018548 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545227ba-ec9a-4683-9e0d-736b5365e69e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.032892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789195-e123-4886-a9bc-1db0cfd8e633 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.060122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fa80e-1d76-4b17-950f-1b6fba25ce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.060766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fa80e-1d76-4b17-950f-1b6fba25ce96 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.061043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fa80e-1d76-4b17-950f-1b6fba25ce96 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.061316 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fa80e-1d76-4b17-950f-1b6fba25ce96 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.070896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fa80e-1d76-4b17-950f-1b6fba25ce96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.072736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fa80e-1d76-4b17-950f-1b6fba25ce96 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.080333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622a3fa-2fa3-4f25-8dc5-19183e055877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.081097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622a3fa-2fa3-4f25-8dc5-19183e055877 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.081400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622a3fa-2fa3-4f25-8dc5-19183e055877 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.081617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622a3fa-2fa3-4f25-8dc5-19183e055877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.085388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e98fa80e-1d76-4b17-950f-1b6fba25ce96 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:11.087885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:11.088848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:11.089885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 406/822] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:11.090532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fa80e-1d76-4b17-950f-1b6fba25ce96 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:11.091567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622a3fa-2fa3-4f25-8dc5-19183e055877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.093328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622a3fa-2fa3-4f25-8dc5-19183e055877 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e622a3fa-2fa3-4f25-8dc5-19183e055877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:11.106972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:11.108104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:11.109199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:11.109199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:11.109199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:11.109199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:11.109199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:11.109199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:11.109199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:11.109199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:11.110943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 417/823] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:11.111873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622a3fa-2fa3-4f25-8dc5-19183e055877 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:11.206095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:11.221823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fa80e-1d76-4b17-950f-1b6fba25ce96 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.249490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622a3fa-2fa3-4f25-8dc5-19183e055877 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.268614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df70a5-c392-41c9-8b6e-0e42e3d5bc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.269749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df70a5-c392-41c9-8b6e-0e42e3d5bc00 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.270171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df70a5-c392-41c9-8b6e-0e42e3d5bc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.270819 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df70a5-c392-41c9-8b6e-0e42e3d5bc00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.272736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df70a5-c392-41c9-8b6e-0e42e3d5bc00 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.274384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df70a5-c392-41c9-8b6e-0e42e3d5bc00 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.295380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be6f12-eb44-4d29-b29e-8de1beb66d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.296171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be6f12-eb44-4d29-b29e-8de1beb66d71 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.296575 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be6f12-eb44-4d29-b29e-8de1beb66d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.297087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be6f12-eb44-4d29-b29e-8de1beb66d71 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.298163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be6f12-eb44-4d29-b29e-8de1beb66d71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:11.298547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be6f12-eb44-4d29-b29e-8de1beb66d71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.300158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be6f12-eb44-4d29-b29e-8de1beb66d71 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.301805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 407/824] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:24:11 2022] POST /identity/v3/endpoints => generated 339 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:11.303363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df70a5-c392-41c9-8b6e-0e42e3d5bc00 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:11.328181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 418/825] 10.222.0.100 () {56 vars in 1154 bytes} [Thu Sep 1 17:24:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:11.334688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be6f12-eb44-4d29-b29e-8de1beb66d71 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:11.397476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:11.435573 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df70a5-c392-41c9-8b6e-0e42e3d5bc00 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.447961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be6f12-eb44-4d29-b29e-8de1beb66d71 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.472549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75d42e-e7c4-42fb-9f1d-95cd8e02ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.473493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75d42e-e7c4-42fb-9f1d-95cd8e02ca2b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.473990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75d42e-e7c4-42fb-9f1d-95cd8e02ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.474438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75d42e-e7c4-42fb-9f1d-95cd8e02ca2b None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.476602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75d42e-e7c4-42fb-9f1d-95cd8e02ca2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:11.477112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75d42e-e7c4-42fb-9f1d-95cd8e02ca2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.478789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75d42e-e7c4-42fb-9f1d-95cd8e02ca2b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.488792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383e326-0075-40cb-88b9-d8c6aaf30958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.489811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383e326-0075-40cb-88b9-d8c6aaf30958 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.490012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383e326-0075-40cb-88b9-d8c6aaf30958 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.490391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383e326-0075-40cb-88b9-d8c6aaf30958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.494157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 408/826] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:11.500017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75d42e-e7c4-42fb-9f1d-95cd8e02ca2b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:11.501971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383e326-0075-40cb-88b9-d8c6aaf30958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.503348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:11.504474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383e326-0075-40cb-88b9-d8c6aaf30958 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b383e326-0075-40cb-88b9-d8c6aaf30958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:11.525665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:11.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:11.528232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:11.528232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:11.528232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:11.529189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 419/827] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:11.534965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383e326-0075-40cb-88b9-d8c6aaf30958 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:11.617175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75d42e-e7c4-42fb-9f1d-95cd8e02ca2b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.648764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383e326-0075-40cb-88b9-d8c6aaf30958 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.655943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba577-4314-486c-b61a-055a4c39d359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.656900 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:11.657735 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba577-4314-486c-b61a-055a4c39d359 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.658156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba577-4314-486c-b61a-055a4c39d359 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.658580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba577-4314-486c-b61a-055a4c39d359 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/f3fa2499f5594b7798f6fe4443a14f4c/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.661019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ba577-4314-486c-b61a-055a4c39d359 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=f3fa2499f5594b7798f6fe4443a14f4c, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.662863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ba577-4314-486c-b61a-055a4c39d359 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.683290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 409/828] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:24:11 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/f3fa2499f5594b7798f6fe4443a14f4c/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:24:11.690715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a2db9-eaea-4828-ab53-be70b05be583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.690715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a2db9-eaea-4828-ab53-be70b05be583 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.690999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a2db9-eaea-4828-ab53-be70b05be583 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.691406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a2db9-eaea-4828-ab53-be70b05be583 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.692156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a2db9-eaea-4828-ab53-be70b05be583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:11.692532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a2db9-eaea-4828-ab53-be70b05be583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.695342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a2db9-eaea-4828-ab53-be70b05be583 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.710870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 420/829] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:11.717094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba577-4314-486c-b61a-055a4c39d359 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:11.798977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:11.850251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba577-4314-486c-b61a-055a4c39d359 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:11.888774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405eb7e3-a99a-4811-9f68-69249af34d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:11.889404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405eb7e3-a99a-4811-9f68-69249af34d91 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:11.889708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405eb7e3-a99a-4811-9f68-69249af34d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:11.889985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405eb7e3-a99a-4811-9f68-69249af34d91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:11.897264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405eb7e3-a99a-4811-9f68-69249af34d91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:11.898845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405eb7e3-a99a-4811-9f68-69249af34d91 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-405eb7e3-a99a-4811-9f68-69249af34d91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:11.910937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:11.911895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:11.912867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:11.912867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:11.912867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:11.912867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:11.912867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:11.912867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:11.912867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:11.912867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:11.912867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 410/830] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:11.918243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a2db9-eaea-4828-ab53-be70b05be583 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:12.032093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a2db9-eaea-4828-ab53-be70b05be583 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:12.067805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878448d-3980-4ff0-9d30-fd48a667098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:12.068379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878448d-3980-4ff0-9d30-fd48a667098c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:12.069157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878448d-3980-4ff0-9d30-fd48a667098c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:12.069427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878448d-3980-4ff0-9d30-fd48a667098c None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:12.070209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878448d-3980-4ff0-9d30-fd48a667098c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:12.070514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878448d-3980-4ff0-9d30-fd48a667098c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:12.072007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878448d-3980-4ff0-9d30-fd48a667098c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:12.084269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 421/831] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:12.090251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405eb7e3-a99a-4811-9f68-69249af34d91 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:12.198918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405eb7e3-a99a-4811-9f68-69249af34d91 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:12.222532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:12.235573 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4f18-221b-4ddd-b2de-e8eb097f3d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:12.236477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4f18-221b-4ddd-b2de-e8eb097f3d73 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:12.237121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4f18-221b-4ddd-b2de-e8eb097f3d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:12.237504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4f18-221b-4ddd-b2de-e8eb097f3d73 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/71b9335f3f744a31a84ab278d9b45e36/roles/3e624c5fdefb4c03bd915bbc9d097212` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:12.254046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4f18-221b-4ddd-b2de-e8eb097f3d73 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=71b9335f3f744a31a84ab278d9b45e36, role_id=3e624c5fdefb4c03bd915bbc9d097212)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:12.255654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4f18-221b-4ddd-b2de-e8eb097f3d73 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:12.270237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 411/832] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:24:12 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/71b9335f3f744a31a84ab278d9b45e36/roles/3e624c5fdefb4c03bd915bbc9d097212 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:24:12.711751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5e611-6374-4101-bb1f-bc6d2d9cd856 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:12.712323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5e611-6374-4101-bb1f-bc6d2d9cd856 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:12.712323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5e611-6374-4101-bb1f-bc6d2d9cd856 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:12.713833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 422/833] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:12.719482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a552-e8d4-4e4b-ac37-4ef92aac9d74 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:12.719893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a552-e8d4-4e4b-ac37-4ef92aac9d74 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:12.720274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a552-e8d4-4e4b-ac37-4ef92aac9d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:12.739467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6946a552-e8d4-4e4b-ac37-4ef92aac9d74 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:12.740803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6946a552-e8d4-4e4b-ac37-4ef92aac9d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:12.825707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 412/834] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:12 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:13.042576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af634f-183f-453a-86a6-5df12b412217 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.042773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af634f-183f-453a-86a6-5df12b412217 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.043031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af634f-183f-453a-86a6-5df12b412217 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.044628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 423/835] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:13.049704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8f81-12e7-4a0e-ba3f-b77b0f2ac10f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.049991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8f81-12e7-4a0e-ba3f-b77b0f2ac10f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.050262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8f81-12e7-4a0e-ba3f-b77b0f2ac10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.068181 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d94f8f81-12e7-4a0e-ba3f-b77b0f2ac10f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:13.070233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d94f8f81-12e7-4a0e-ba3f-b77b0f2ac10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:13.094372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af634f-183f-453a-86a6-5df12b412217 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:13.151360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 413/836] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:13 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:13.152305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:13.203226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af634f-183f-453a-86a6-5df12b412217 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:13.237794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fb2fa-3949-4628-903d-14a58b7e0710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:13.238741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fb2fa-3949-4628-903d-14a58b7e0710 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.239196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fb2fa-3949-4628-903d-14a58b7e0710 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.239638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fb2fa-3949-4628-903d-14a58b7e0710 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.240434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fb2fa-3949-4628-903d-14a58b7e0710 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:13.241999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fb2fa-3949-4628-903d-14a58b7e0710 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8b5fb2fa-3949-4628-903d-14a58b7e0710 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:13.251156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:13.252129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:24:13.253020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:13.254949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 424/837] 10.222.0.100 () {56 vars in 1137 bytes} [Thu Sep 1 17:24:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:13.266699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8f81-12e7-4a0e-ba3f-b77b0f2ac10f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:13.300155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:13.373265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8f81-12e7-4a0e-ba3f-b77b0f2ac10f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:13.409504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa974c-5b35-4318-96e6-3ec415afdc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:13.410185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa974c-5b35-4318-96e6-3ec415afdc18 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.410465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa974c-5b35-4318-96e6-3ec415afdc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.410741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa974c-5b35-4318-96e6-3ec415afdc18 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.411401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa974c-5b35-4318-96e6-3ec415afdc18 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:13.411711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa974c-5b35-4318-96e6-3ec415afdc18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:13.413843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa974c-5b35-4318-96e6-3ec415afdc18 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:13.419587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 414/838] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:13 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:13.425103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fb2fa-3949-4628-903d-14a58b7e0710 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:13.442732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa974c-5b35-4318-96e6-3ec415afdc18 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:13.528605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:13.532984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fb2fa-3949-4628-903d-14a58b7e0710 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:13.563737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:13.568991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723c92b-62c1-45b0-8dcc-66f2bb1c14da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:13.569937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723c92b-62c1-45b0-8dcc-66f2bb1c14da None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.570068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa974c-5b35-4318-96e6-3ec415afdc18 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:13.570555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723c92b-62c1-45b0-8dcc-66f2bb1c14da None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.571037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723c92b-62c1-45b0-8dcc-66f2bb1c14da None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.571931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723c92b-62c1-45b0-8dcc-66f2bb1c14da None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:13.572479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723c92b-62c1-45b0-8dcc-66f2bb1c14da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:13.574216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723c92b-62c1-45b0-8dcc-66f2bb1c14da None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:13.585938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 425/839] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:13 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:13.610384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef249c5-8e31-4d41-b86a-9300e3121633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:13.611028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef249c5-8e31-4d41-b86a-9300e3121633 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.611307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef249c5-8e31-4d41-b86a-9300e3121633 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.611575 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef249c5-8e31-4d41-b86a-9300e3121633 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.612165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef249c5-8e31-4d41-b86a-9300e3121633 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:13.613563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef249c5-8e31-4d41-b86a-9300e3121633 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9ef249c5-8e31-4d41-b86a-9300e3121633 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:13.624148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:13.625219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:24:13.626076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:13.630235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 415/840] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:24:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:13.631684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723c92b-62c1-45b0-8dcc-66f2bb1c14da None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:13.718853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4f6ae-adbc-44e6-a231-de2a927f1881 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.719216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4f6ae-adbc-44e6-a231-de2a927f1881 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.719396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4f6ae-adbc-44e6-a231-de2a927f1881 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.721109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 416/841] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:13.727988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b209d90-e63a-4431-b6bf-11a991306547 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.728274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b209d90-e63a-4431-b6bf-11a991306547 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.729011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b209d90-e63a-4431-b6bf-11a991306547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.753167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6b209d90-e63a-4431-b6bf-11a991306547 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:13.754196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723c92b-62c1-45b0-8dcc-66f2bb1c14da None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:13.754622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6b209d90-e63a-4431-b6bf-11a991306547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:13.800227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0ab6a-b60b-4636-8459-5b4f0ea5dcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:13.800838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0ab6a-b60b-4636-8459-5b4f0ea5dcca None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.801118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0ab6a-b60b-4636-8459-5b4f0ea5dcca None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.801391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0ab6a-b60b-4636-8459-5b4f0ea5dcca None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.802056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0ab6a-b60b-4636-8459-5b4f0ea5dcca None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:13.802356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0ab6a-b60b-4636-8459-5b4f0ea5dcca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:13.804148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0ab6a-b60b-4636-8459-5b4f0ea5dcca None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:13.810214 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 426/842] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:24:13 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:13.815675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0ab6a-b60b-4636-8459-5b4f0ea5dcca None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:13.836577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:13.846934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 417/843] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:13 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:13.929022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0ab6a-b60b-4636-8459-5b4f0ea5dcca None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:13.965898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26705222-9a25-421d-a1fc-c248606e82b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:13.966873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26705222-9a25-421d-a1fc-c248606e82b8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:13.967417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26705222-9a25-421d-a1fc-c248606e82b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:13.967890 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26705222-9a25-421d-a1fc-c248606e82b8 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:13.968771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26705222-9a25-421d-a1fc-c248606e82b8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:13.969295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26705222-9a25-421d-a1fc-c248606e82b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:13.970889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26705222-9a25-421d-a1fc-c248606e82b8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:13.980272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 427/844] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:24:13 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:14.147776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b209d90-e63a-4431-b6bf-11a991306547 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:14.244862 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:14.256745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b209d90-e63a-4431-b6bf-11a991306547 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:14.291620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fab49f-c65b-4fb0-97c2-0c14b5998e0b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.291950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fab49f-c65b-4fb0-97c2-0c14b5998e0b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.292205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fab49f-c65b-4fb0-97c2-0c14b5998e0b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.294562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e1771-89fe-496d-afd2-866ab59de30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:14.294792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 428/845] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:14.295603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e1771-89fe-496d-afd2-866ab59de30e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.296089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e1771-89fe-496d-afd2-866ab59de30e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.296564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e1771-89fe-496d-afd2-866ab59de30e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.298806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6334-6ea2-42bd-92f5-208db74a7240 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.299199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6334-6ea2-42bd-92f5-208db74a7240 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.299479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6334-6ea2-42bd-92f5-208db74a7240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.309641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e1771-89fe-496d-afd2-866ab59de30e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8b7e1771-89fe-496d-afd2-866ab59de30e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:14.314977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:14.315934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:14.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:14.319221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-449b6334-6ea2-42bd-92f5-208db74a7240 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:14.320241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 418/846] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:14.324995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-449b6334-6ea2-42bd-92f5-208db74a7240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:14.326428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e1771-89fe-496d-afd2-866ab59de30e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:14.391610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 429/847] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:14 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:14.436595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e1771-89fe-496d-afd2-866ab59de30e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:14.471896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a5c4b-b179-48e0-a9a3-a3cb0802f763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:14.472873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a5c4b-b179-48e0-a9a3-a3cb0802f763 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.473354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a5c4b-b179-48e0-a9a3-a3cb0802f763 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.473910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a5c4b-b179-48e0-a9a3-a3cb0802f763 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.475332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a5c4b-b179-48e0-a9a3-a3cb0802f763 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:14.475867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a5c4b-b179-48e0-a9a3-a3cb0802f763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:14.477513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a5c4b-b179-48e0-a9a3-a3cb0802f763 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:14.486954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 419/848] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:24:14 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:14.492970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6334-6ea2-42bd-92f5-208db74a7240 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:14.508966 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:14.614200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6334-6ea2-42bd-92f5-208db74a7240 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:14.655310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba28c1-162a-4272-a6f6-bc235eac5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:14.656231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba28c1-162a-4272-a6f6-bc235eac5212 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.656718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba28c1-162a-4272-a6f6-bc235eac5212 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.662887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba28c1-162a-4272-a6f6-bc235eac5212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.679423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba28c1-162a-4272-a6f6-bc235eac5212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:14.681274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba28c1-162a-4272-a6f6-bc235eac5212 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:14.695216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a5c4b-b179-48e0-a9a3-a3cb0802f763 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4bba28c1-162a-4272-a6f6-bc235eac5212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:14.704170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:14.705157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:14.706119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:14.706119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:14.706119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:14.712721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 430/849] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:14.713418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba28c1-162a-4272-a6f6-bc235eac5212 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:14.718328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:14.792052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:14.834672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba28c1-162a-4272-a6f6-bc235eac5212 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:14.835898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a5c4b-b179-48e0-a9a3-a3cb0802f763 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:14.877070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6e131-3438-4c12-a274-252299eb39db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:14.877881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6e131-3438-4c12-a274-252299eb39db None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.878228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6e131-3438-4c12-a274-252299eb39db None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.878609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6e131-3438-4c12-a274-252299eb39db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.879057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9734fd-9b72-4869-8db6-d836cab8a6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:14.880019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9734fd-9b72-4869-8db6-d836cab8a6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:14.880804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9734fd-9b72-4869-8db6-d836cab8a6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:14.881344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9734fd-9b72-4869-8db6-d836cab8a6d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:14.882571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9734fd-9b72-4869-8db6-d836cab8a6d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:14.883059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9734fd-9b72-4869-8db6-d836cab8a6d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:14.884925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9734fd-9b72-4869-8db6-d836cab8a6d5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:14.890925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6e131-3438-4c12-a274-252299eb39db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-59f6e131-3438-4c12-a274-252299eb39db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:14.898297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:14.899263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:24:14.900118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:14.907825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 420/851] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:24:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:14.907825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6e131-3438-4c12-a274-252299eb39db None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:14.908187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 431/851] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:14.909493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9734fd-9b72-4869-8db6-d836cab8a6d5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:14.987074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:15.020941 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6e131-3438-4c12-a274-252299eb39db None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:15.054554 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9734fd-9b72-4869-8db6-d836cab8a6d5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:15.059121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a74f3-cb50-4e18-b4aa-296839f66e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:15.059725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a74f3-cb50-4e18-b4aa-296839f66e0d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.060001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a74f3-cb50-4e18-b4aa-296839f66e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.060271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a74f3-cb50-4e18-b4aa-296839f66e0d None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.060995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a74f3-cb50-4e18-b4aa-296839f66e0d None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:15.061294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a74f3-cb50-4e18-b4aa-296839f66e0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:15.062736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a74f3-cb50-4e18-b4aa-296839f66e0d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:15.073656 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 421/852] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:24:14 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:15.079136 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a74f3-cb50-4e18-b4aa-296839f66e0d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:15.094561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:15.111536 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64968226-eab0-4299-a70e-5671a9060ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:15.113417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64968226-eab0-4299-a70e-5671a9060ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.113961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64968226-eab0-4299-a70e-5671a9060ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.114803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64968226-eab0-4299-a70e-5671a9060ba7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.125991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64968226-eab0-4299-a70e-5671a9060ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:15.128273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64968226-eab0-4299-a70e-5671a9060ba7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-64968226-eab0-4299-a70e-5671a9060ba7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:15.145532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:15.147085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:24:15.148134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:15.156382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 432/853] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:15.156882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64968226-eab0-4299-a70e-5671a9060ba7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:15.199018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a74f3-cb50-4e18-b4aa-296839f66e0d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:15.241502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bda47-62e2-424d-8160-823424fbb260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:15.242521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bda47-62e2-424d-8160-823424fbb260 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.243237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bda47-62e2-424d-8160-823424fbb260 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.243777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bda47-62e2-424d-8160-823424fbb260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.246865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:15.260909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bda47-62e2-424d-8160-823424fbb260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:15.263340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bda47-62e2-424d-8160-823424fbb260 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-412bda47-62e2-424d-8160-823424fbb260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:15.282760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:15.283992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:15.285101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:15.285101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:15.285101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:15.291175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 422/854] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:15.291821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64968226-eab0-4299-a70e-5671a9060ba7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:15.292632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bda47-62e2-424d-8160-823424fbb260 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:15.340234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc30b1-e8bf-4e32-a108-c6697c600fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:15.341354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc30b1-e8bf-4e32-a108-c6697c600fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.341834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc30b1-e8bf-4e32-a108-c6697c600fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.342287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc30b1-e8bf-4e32-a108-c6697c600fc3 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.343912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc30b1-e8bf-4e32-a108-c6697c600fc3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:15.344390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc30b1-e8bf-4e32-a108-c6697c600fc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:15.346531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc30b1-e8bf-4e32-a108-c6697c600fc3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:15.377894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 433/855] 10.222.0.100 () {56 vars in 1140 bytes} [Thu Sep 1 17:24:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:15.378570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc30b1-e8bf-4e32-a108-c6697c600fc3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:15.420881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bda47-62e2-424d-8160-823424fbb260 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:15.429097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:15.470049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d9e78-836c-4ed7-bac7-d82d71cd4901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:15.471287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d9e78-836c-4ed7-bac7-d82d71cd4901 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.472239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d9e78-836c-4ed7-bac7-d82d71cd4901 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.472965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d9e78-836c-4ed7-bac7-d82d71cd4901 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.473869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d9e78-836c-4ed7-bac7-d82d71cd4901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:15.475181 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d9e78-836c-4ed7-bac7-d82d71cd4901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:15.477355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d9e78-836c-4ed7-bac7-d82d71cd4901 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:15.493080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc30b1-e8bf-4e32-a108-c6697c600fc3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:15.497270 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 423/856] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:15.506597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d9e78-836c-4ed7-bac7-d82d71cd4901 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:15.543088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fc9dc-e951-4328-8428-1d94de8e822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:15.544984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fc9dc-e951-4328-8428-1d94de8e822e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.545493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fc9dc-e951-4328-8428-1d94de8e822e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.546028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fc9dc-e951-4328-8428-1d94de8e822e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.557549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:15.620339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d9e78-836c-4ed7-bac7-d82d71cd4901 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:15.627024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fc9dc-e951-4328-8428-1d94de8e822e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:15.628464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fc9dc-e951-4328-8428-1d94de8e822e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:15.660379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d70f3-e2eb-42aa-81d5-80cb321068ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:15.661345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d70f3-e2eb-42aa-81d5-80cb321068ad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.661652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d70f3-e2eb-42aa-81d5-80cb321068ad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.661931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d70f3-e2eb-42aa-81d5-80cb321068ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.670218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d70f3-e2eb-42aa-81d5-80cb321068ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:15.671734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d70f3-e2eb-42aa-81d5-80cb321068ad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:15.681747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d38d70f3-e2eb-42aa-81d5-80cb321068ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:15.686842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:15.687791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:15.688786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 424/857] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:24:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:15.689381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec156b-4a39-4a3a-a27e-841c3fe5eb8c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.689462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec156b-4a39-4a3a-a27e-841c3fe5eb8c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.690184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec156b-4a39-4a3a-a27e-841c3fe5eb8c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.691180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 425/858] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:15 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:15.694789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec156b-4a39-4a3a-a27e-841c3fe5eb8c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:15.747978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 434/859] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:15 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:24:15.749213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae38fce-d079-4c01-a1ef-75dccfd36de4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.749498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae38fce-d079-4c01-a1ef-75dccfd36de4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.749787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae38fce-d079-4c01-a1ef-75dccfd36de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.769330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cae38fce-d079-4c01-a1ef-75dccfd36de4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:15.770559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cae38fce-d079-4c01-a1ef-75dccfd36de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:15.814235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec156b-4a39-4a3a-a27e-841c3fe5eb8c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:15.822918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:15.844719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 435/860] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:15 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:15.854843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae38fce-d079-4c01-a1ef-75dccfd36de4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:15.856573 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb41e0c-60d1-4e06-94f3-ae56733fd297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:15.857370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb41e0c-60d1-4e06-94f3-ae56733fd297 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:15.857840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb41e0c-60d1-4e06-94f3-ae56733fd297 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:15.858298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb41e0c-60d1-4e06-94f3-ae56733fd297 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:15.859307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb41e0c-60d1-4e06-94f3-ae56733fd297 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:15.859843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb41e0c-60d1-4e06-94f3-ae56733fd297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:15.861622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb41e0c-60d1-4e06-94f3-ae56733fd297 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:15.885560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 426/861] 10.222.0.100 () {56 vars in 1154 bytes} [Thu Sep 1 17:24:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:15.887217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb41e0c-60d1-4e06-94f3-ae56733fd297 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:15.945530 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:15.979814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae38fce-d079-4c01-a1ef-75dccfd36de4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:15.995527 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb41e0c-60d1-4e06-94f3-ae56733fd297 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.020955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17f4af-dcfa-4beb-921d-854c744773ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.022101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17f4af-dcfa-4beb-921d-854c744773ea None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.022581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17f4af-dcfa-4beb-921d-854c744773ea None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.022899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17f4af-dcfa-4beb-921d-854c744773ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.032103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd8ab1-4a16-4755-9f85-a203f7de0758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.032724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd8ab1-4a16-4755-9f85-a203f7de0758 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.032996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd8ab1-4a16-4755-9f85-a203f7de0758 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.033251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd8ab1-4a16-4755-9f85-a203f7de0758 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.034097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17f4af-dcfa-4beb-921d-854c744773ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.035698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17f4af-dcfa-4beb-921d-854c744773ea None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8a17f4af-dcfa-4beb-921d-854c744773ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:16.051053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:16.052179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:16.053336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:16.053336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:16.053336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:16.059463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 436/862] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:16.060869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60743df-e0fa-4017-9eda-6862922a5d43 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.061316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60743df-e0fa-4017-9eda-6862922a5d43 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.061758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60743df-e0fa-4017-9eda-6862922a5d43 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.063412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 437/863] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:16.064280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60743df-e0fa-4017-9eda-6862922a5d43 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:16.069624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.112948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd8ab1-4a16-4755-9f85-a203f7de0758 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.114598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd8ab1-4a16-4755-9f85-a203f7de0758 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.172641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60743df-e0fa-4017-9eda-6862922a5d43 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.196647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.207470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483a08f-cd6b-447f-8521-992132eb8884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.208116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483a08f-cd6b-447f-8521-992132eb8884 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.208398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483a08f-cd6b-447f-8521-992132eb8884 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.208665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483a08f-cd6b-447f-8521-992132eb8884 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.210089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483a08f-cd6b-447f-8521-992132eb8884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:16.210383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483a08f-cd6b-447f-8521-992132eb8884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.211745 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483a08f-cd6b-447f-8521-992132eb8884 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.218673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 427/864] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:15 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:24:16.218673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f774-4f38-411a-b43a-04f4f275acf4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.218673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f774-4f38-411a-b43a-04f4f275acf4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.218673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f774-4f38-411a-b43a-04f4f275acf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.223677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 438/865] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:16.224520 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483a08f-cd6b-447f-8521-992132eb8884 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:16.234785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9044f774-4f38-411a-b43a-04f4f275acf4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:16.235968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9044f774-4f38-411a-b43a-04f4f275acf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:16.300052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.309614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 428/866] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:16 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:16.310500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044f774-4f38-411a-b43a-04f4f275acf4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:16.333338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483a08f-cd6b-447f-8521-992132eb8884 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.371868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b8b73-b660-4a59-af82-4ab8da5b3f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.372475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b8b73-b660-4a59-af82-4ab8da5b3f32 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.372750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b8b73-b660-4a59-af82-4ab8da5b3f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.373019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b8b73-b660-4a59-af82-4ab8da5b3f32 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.373691 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b8b73-b660-4a59-af82-4ab8da5b3f32 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.375041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b8b73-b660-4a59-af82-4ab8da5b3f32 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.391927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 439/867] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:24:16 2022] POST /identity/v3/services => generated 270 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:16.391927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b8b73-b660-4a59-af82-4ab8da5b3f32 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:16.437793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044f774-4f38-411a-b43a-04f4f275acf4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.454528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.485165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c73e6-1319-4d2f-9e46-a9bb28d61f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.487267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c73e6-1319-4d2f-9e46-a9bb28d61f2b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.487789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c73e6-1319-4d2f-9e46-a9bb28d61f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.488452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c73e6-1319-4d2f-9e46-a9bb28d61f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.503600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c73e6-1319-4d2f-9e46-a9bb28d61f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.506684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c73e6-1319-4d2f-9e46-a9bb28d61f2b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.524609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b8b73-b660-4a59-af82-4ab8da5b3f32 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-af3c73e6-1319-4d2f-9e46-a9bb28d61f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:16.528950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:16.529991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:16.530888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:16.530888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:16.530888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:16.533815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 429/868] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:16.540006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c73e6-1319-4d2f-9e46-a9bb28d61f2b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:16.573114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc88e7-b808-4f0b-8113-4006fdbf6f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.573828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc88e7-b808-4f0b-8113-4006fdbf6f41 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.574105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc88e7-b808-4f0b-8113-4006fdbf6f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.574375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc88e7-b808-4f0b-8113-4006fdbf6f41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.583091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc88e7-b808-4f0b-8113-4006fdbf6f41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.584807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc88e7-b808-4f0b-8113-4006fdbf6f41 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.595247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-aabc88e7-b808-4f0b-8113-4006fdbf6f41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:16.604524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:16.605480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 440/869] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:16.606406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc88e7-b808-4f0b-8113-4006fdbf6f41 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:16.664662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c73e6-1319-4d2f-9e46-a9bb28d61f2b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.710387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41279b3e-20c4-464a-b96b-bb35494123c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.711320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41279b3e-20c4-464a-b96b-bb35494123c1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.711739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41279b3e-20c4-464a-b96b-bb35494123c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.712146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41279b3e-20c4-464a-b96b-bb35494123c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.712787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41279b3e-20c4-464a-b96b-bb35494123c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:16.713253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41279b3e-20c4-464a-b96b-bb35494123c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.715268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41279b3e-20c4-464a-b96b-bb35494123c1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.716509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc88e7-b808-4f0b-8113-4006fdbf6f41 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.719703 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.738048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 430/870] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:16.738048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41279b3e-20c4-464a-b96b-bb35494123c1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:16.756785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf019fd-bb66-40dc-b817-26f5a3cd82b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.757484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf019fd-bb66-40dc-b817-26f5a3cd82b8 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.757736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf019fd-bb66-40dc-b817-26f5a3cd82b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.758011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf019fd-bb66-40dc-b817-26f5a3cd82b8 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.758673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf019fd-bb66-40dc-b817-26f5a3cd82b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.760035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf019fd-bb66-40dc-b817-26f5a3cd82b8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.785189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 441/871] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:24:16 2022] POST /identity/v3/services => generated 252 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:16.785189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf019fd-bb66-40dc-b817-26f5a3cd82b8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:16.881695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41279b3e-20c4-464a-b96b-bb35494123c1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.883996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:16.947060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd290536-992d-4c9f-a2ac-6615e7be3123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:16.947060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd290536-992d-4c9f-a2ac-6615e7be3123 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:16.947060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd290536-992d-4c9f-a2ac-6615e7be3123 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:16.947060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd290536-992d-4c9f-a2ac-6615e7be3123 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:16.949063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd290536-992d-4c9f-a2ac-6615e7be3123 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:16.949063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd290536-992d-4c9f-a2ac-6615e7be3123 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:16.950698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd290536-992d-4c9f-a2ac-6615e7be3123 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:16.954423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf019fd-bb66-40dc-b817-26f5a3cd82b8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:16.966294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 431/872] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:16.973068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd290536-992d-4c9f-a2ac-6615e7be3123 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:17.004357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc755a-6d55-4280-82d5-5d5b5d6c82e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:17.005004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc755a-6d55-4280-82d5-5d5b5d6c82e9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:17.005287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc755a-6d55-4280-82d5-5d5b5d6c82e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:17.005563 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc755a-6d55-4280-82d5-5d5b5d6c82e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:17.013437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc755a-6d55-4280-82d5-5d5b5d6c82e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:17.015028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc755a-6d55-4280-82d5-5d5b5d6c82e9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-70cc755a-6d55-4280-82d5-5d5b5d6c82e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:17.028009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:17.028970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:17.029973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:17.029973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:17.029973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:17.029973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:17.029973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:17.029973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:17.029973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:17.029973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:17.031953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 442/873] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:17.038436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc755a-6d55-4280-82d5-5d5b5d6c82e9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:17.075447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:17.102690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd290536-992d-4c9f-a2ac-6615e7be3123 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:17.140626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae864d-6f1a-467b-aafb-aaa3a1796747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:17.141065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae864d-6f1a-467b-aafb-aaa3a1796747 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:17.141413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae864d-6f1a-467b-aafb-aaa3a1796747 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:17.141818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae864d-6f1a-467b-aafb-aaa3a1796747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:17.142643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae864d-6f1a-467b-aafb-aaa3a1796747 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=f3fa2499f5594b7798f6fe4443a14f4c) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:17.142982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae864d-6f1a-467b-aafb-aaa3a1796747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:17.144528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae864d-6f1a-467b-aafb-aaa3a1796747 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:17.157601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 432/874] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:16 2022] GET /identity/v3/role_assignments?user.id=f3fa2499f5594b7798f6fe4443a14f4c&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 610 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:17.189500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc755a-6d55-4280-82d5-5d5b5d6c82e9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:17.228391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed0523-308c-4bb4-aeab-f00ccdc96e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:17.229078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0523-308c-4bb4-aeab-f00ccdc96e08 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:17.229363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0523-308c-4bb4-aeab-f00ccdc96e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:17.229666 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0523-308c-4bb4-aeab-f00ccdc96e08 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:17.230428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed0523-308c-4bb4-aeab-f00ccdc96e08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:17.230740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed0523-308c-4bb4-aeab-f00ccdc96e08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:17.232289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed0523-308c-4bb4-aeab-f00ccdc96e08 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:17.246916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 443/875] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:17.253051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae864d-6f1a-467b-aafb-aaa3a1796747 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:17.282073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:17.369849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae864d-6f1a-467b-aafb-aaa3a1796747 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:17.408627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1fffb-f6f9-4315-b6d6-f55934c44e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:17.409495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1fffb-f6f9-4315-b6d6-f55934c44e72 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:17.410010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1fffb-f6f9-4315-b6d6-f55934c44e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:17.410459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1fffb-f6f9-4315-b6d6-f55934c44e72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:17.411340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1fffb-f6f9-4315-b6d6-f55934c44e72 None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=71b9335f3f744a31a84ab278d9b45e36) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:17.411807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1fffb-f6f9-4315-b6d6-f55934c44e72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:17.414088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1fffb-f6f9-4315-b6d6-f55934c44e72 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:17.421098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 433/876] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:17 2022] GET /identity/v3/role_assignments?user.id=71b9335f3f744a31a84ab278d9b45e36&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 610 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:18.399630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be24d-f907-45fc-84b6-c7599d06d1ef None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:18.400152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be24d-f907-45fc-84b6-c7599d06d1ef None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:18.400152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be24d-f907-45fc-84b6-c7599d06d1ef None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:18.401669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 444/877] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:18.408245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859224-4649-4eee-87cc-2b4a53ea55b5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:18.408673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859224-4649-4eee-87cc-2b4a53ea55b5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:18.408842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859224-4649-4eee-87cc-2b4a53ea55b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:18.429035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d9859224-4649-4eee-87cc-2b4a53ea55b5 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:18.430261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d9859224-4649-4eee-87cc-2b4a53ea55b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:18.503184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 434/878] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:18 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:18.775909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9be24d-f907-45fc-84b6-c7599d06d1ef None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:18.837632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3763521-85cf-4976-903a-f796538a3567 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:18.837955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3763521-85cf-4976-903a-f796538a3567 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:18.838199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3763521-85cf-4976-903a-f796538a3567 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:18.839815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 435/879] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:18.845616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2405df-d48b-4e27-ad02-ef1b96244e3a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:18.845616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2405df-d48b-4e27-ad02-ef1b96244e3a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:18.845616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2405df-d48b-4e27-ad02-ef1b96244e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:18.864821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3b2405df-d48b-4e27-ad02-ef1b96244e3a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:18.866609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3b2405df-d48b-4e27-ad02-ef1b96244e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:18.874724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:18.883700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:18.893915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9be24d-f907-45fc-84b6-c7599d06d1ef None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:18.932395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498188-d3d5-4c18-aef8-d16e2b1df4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:18.932956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498188-d3d5-4c18-aef8-d16e2b1df4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:18.933240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498188-d3d5-4c18-aef8-d16e2b1df4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:18.933606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498188-d3d5-4c18-aef8-d16e2b1df4d7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:18.934129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7498188-d3d5-4c18-aef8-d16e2b1df4d7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:18.935510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7498188-d3d5-4c18-aef8-d16e2b1df4d7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a7498188-d3d5-4c18-aef8-d16e2b1df4d7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:18.946199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:18.947202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:18.948125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 445/880] 10.222.0.100 () {56 vars in 1137 bytes} [Thu Sep 1 17:24:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:18.954498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498188-d3d5-4c18-aef8-d16e2b1df4d7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:18.965666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 436/881] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:18 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:19.068302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498188-d3d5-4c18-aef8-d16e2b1df4d7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.103952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee42b08-ca8b-4d2f-868a-cefde27a3693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:19.104570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee42b08-ca8b-4d2f-868a-cefde27a3693 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.104851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee42b08-ca8b-4d2f-868a-cefde27a3693 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.105123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee42b08-ca8b-4d2f-868a-cefde27a3693 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.105788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee42b08-ca8b-4d2f-868a-cefde27a3693 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:19.106096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee42b08-ca8b-4d2f-868a-cefde27a3693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:19.107522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee42b08-ca8b-4d2f-868a-cefde27a3693 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:19.112890 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 446/882] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:18 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:19.118919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2405df-d48b-4e27-ad02-ef1b96244e3a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.227260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2405df-d48b-4e27-ad02-ef1b96244e3a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.247266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee42b08-ca8b-4d2f-868a-cefde27a3693 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.265232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:19.267060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab995e36-6f7f-4b2c-ac84-c71382ccdb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:19.267728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab995e36-6f7f-4b2c-ac84-c71382ccdb0c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.267976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab995e36-6f7f-4b2c-ac84-c71382ccdb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.268222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab995e36-6f7f-4b2c-ac84-c71382ccdb0c None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.268854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab995e36-6f7f-4b2c-ac84-c71382ccdb0c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:19.269157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab995e36-6f7f-4b2c-ac84-c71382ccdb0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:19.270657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab995e36-6f7f-4b2c-ac84-c71382ccdb0c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:19.275983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 437/883] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:19.277257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df385cd-4caf-4411-85c2-5a5328370bf6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.277541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df385cd-4caf-4411-85c2-5a5328370bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.278106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df385cd-4caf-4411-85c2-5a5328370bf6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.283188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 438/884] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:19.283188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df385cd-4caf-4411-85c2-5a5328370bf6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.295495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:19.370433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee42b08-ca8b-4d2f-868a-cefde27a3693 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.394622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df385cd-4caf-4411-85c2-5a5328370bf6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.407739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f6e34-0e25-42c5-b261-403dbf54aa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:19.408399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f6e34-0e25-42c5-b261-403dbf54aa29 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.408641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f6e34-0e25-42c5-b261-403dbf54aa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.409023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f6e34-0e25-42c5-b261-403dbf54aa29 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.409747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f6e34-0e25-42c5-b261-403dbf54aa29 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:19.411103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f6e34-0e25-42c5-b261-403dbf54aa29 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-406f6e34-0e25-42c5-b261-403dbf54aa29 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:19.421359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:19.422468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:19.423674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 447/885] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:19.424511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0a53a-ad26-42ec-a86d-7b81f95b2310 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.424511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0a53a-ad26-42ec-a86d-7b81f95b2310 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.424511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0a53a-ad26-42ec-a86d-7b81f95b2310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.436022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa6601-4017-4197-8bac-1f505b45f252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:19.436649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa6601-4017-4197-8bac-1f505b45f252 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.436900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa6601-4017-4197-8bac-1f505b45f252 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.437169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa6601-4017-4197-8bac-1f505b45f252 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.437888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa6601-4017-4197-8bac-1f505b45f252 None admin] RBAC: Adding query filter params (interface=public, service_id=69cd58ae001d4b228d3bdb4de56b5c2e, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:19.438193 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa6601-4017-4197-8bac-1f505b45f252 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:19.439544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa6601-4017-4197-8bac-1f505b45f252 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:19.444761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-49f0a53a-ad26-42ec-a86d-7b81f95b2310 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:19.445482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 439/886] 10.222.0.100 () {56 vars in 1256 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/endpoints?service_id=69cd58ae001d4b228d3bdb4de56b5c2e&interface=public®ion_id=RegionOne => generated 195 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:19.446096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-49f0a53a-ad26-42ec-a86d-7b81f95b2310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:19.450508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa6601-4017-4197-8bac-1f505b45f252 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.514311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 448/887] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:19 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:19.515058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0a53a-ad26-42ec-a86d-7b81f95b2310 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.525993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:19.529675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:19.557029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa6601-4017-4197-8bac-1f505b45f252 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.596691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942945b-43e2-43b8-a09a-dfe36fc329da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:19.597323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942945b-43e2-43b8-a09a-dfe36fc329da None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.597621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942945b-43e2-43b8-a09a-dfe36fc329da None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.597893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942945b-43e2-43b8-a09a-dfe36fc329da None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.598516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942945b-43e2-43b8-a09a-dfe36fc329da None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:19.598811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942945b-43e2-43b8-a09a-dfe36fc329da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:19.600407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942945b-43e2-43b8-a09a-dfe36fc329da None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:19.609463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 440/888] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:19.615817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b3ca0-fbe4-454b-9dcd-f7b95342bd36 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.616227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b3ca0-fbe4-454b-9dcd-f7b95342bd36 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.616499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b3ca0-fbe4-454b-9dcd-f7b95342bd36 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.617979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 441/889] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:19.618482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b3ca0-fbe4-454b-9dcd-f7b95342bd36 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.634450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0a53a-ad26-42ec-a86d-7b81f95b2310 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.676466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c5bf4-aa80-4473-b08d-a3544fe28ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:19.677075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c5bf4-aa80-4473-b08d-a3544fe28ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.677357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c5bf4-aa80-4473-b08d-a3544fe28ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.678439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c5bf4-aa80-4473-b08d-a3544fe28ed4 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.679053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c5bf4-aa80-4473-b08d-a3544fe28ed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:19.680412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c5bf4-aa80-4473-b08d-a3544fe28ed4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:19.686208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 449/890] 10.222.0.100 () {56 vars in 1092 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/services => generated 1274 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:24:19.687534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010e662-5482-4ee3-b2c3-c238bb95f9fa None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.687896 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010e662-5482-4ee3-b2c3-c238bb95f9fa None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.688243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010e662-5482-4ee3-b2c3-c238bb95f9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.708206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d010e662-5482-4ee3-b2c3-c238bb95f9fa None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:19.709656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d010e662-5482-4ee3-b2c3-c238bb95f9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:19.753072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b3ca0-fbe4-454b-9dcd-f7b95342bd36 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.782958 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:19.789662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 450/891] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:19 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:19.790711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010e662-5482-4ee3-b2c3-c238bb95f9fa None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.799146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340d078-b8ac-4e20-a705-bf58d04025da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:19.800189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340d078-b8ac-4e20-a705-bf58d04025da None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.800683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340d078-b8ac-4e20-a705-bf58d04025da None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.801131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340d078-b8ac-4e20-a705-bf58d04025da None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.802054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340d078-b8ac-4e20-a705-bf58d04025da None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:19.803456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340d078-b8ac-4e20-a705-bf58d04025da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:19.805198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340d078-b8ac-4e20-a705-bf58d04025da None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:19.816641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 442/892] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:19.817919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340d078-b8ac-4e20-a705-bf58d04025da None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.821353 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:19.909825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010e662-5482-4ee3-b2c3-c238bb95f9fa None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.957075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340d078-b8ac-4e20-a705-bf58d04025da None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:19.958046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7f4f5-a55a-4c97-87c8-96f50621a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:19.958923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7f4f5-a55a-4c97-87c8-96f50621a69b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:19.959506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7f4f5-a55a-4c97-87c8-96f50621a69b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:19.960082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7f4f5-a55a-4c97-87c8-96f50621a69b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:19.960856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7f4f5-a55a-4c97-87c8-96f50621a69b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:19.962478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7f4f5-a55a-4c97-87c8-96f50621a69b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e9d7f4f5-a55a-4c97-87c8-96f50621a69b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:19.971926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:19.972933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:19.973800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:19.977108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 451/893] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:19.982564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7f4f5-a55a-4c97-87c8-96f50621a69b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:19.999449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595ac94-0d68-4f97-898c-f49a7bca578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:20.004782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595ac94-0d68-4f97-898c-f49a7bca578c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:20.004782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595ac94-0d68-4f97-898c-f49a7bca578c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:20.004782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595ac94-0d68-4f97-898c-f49a7bca578c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:20.008132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595ac94-0d68-4f97-898c-f49a7bca578c None admin] RBAC: Adding query filter params (interface=public, service_id=2c2b4e72dcc14fa8b7daf502562ce820, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:20.008132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595ac94-0d68-4f97-898c-f49a7bca578c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:20.008132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595ac94-0d68-4f97-898c-f49a7bca578c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:20.014559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 443/894] 10.222.0.100 () {56 vars in 1256 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/endpoints?service_id=2c2b4e72dcc14fa8b7daf502562ce820&interface=public®ion_id=RegionOne => generated 195 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:20.020490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595ac94-0d68-4f97-898c-f49a7bca578c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:20.068501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:20.105133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:20.111078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7f4f5-a55a-4c97-87c8-96f50621a69b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:20.142971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595ac94-0d68-4f97-898c-f49a7bca578c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:20.148769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5f8d9-828b-4641-af30-bd0eec2be098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:20.149371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5f8d9-828b-4641-af30-bd0eec2be098 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:20.149674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5f8d9-828b-4641-af30-bd0eec2be098 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:20.149949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5f8d9-828b-4641-af30-bd0eec2be098 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:20.150579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5f8d9-828b-4641-af30-bd0eec2be098 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:20.150879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5f8d9-828b-4641-af30-bd0eec2be098 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:20.152376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5f8d9-828b-4641-af30-bd0eec2be098 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:20.157308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 452/895] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:19 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:20.163398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5f8d9-828b-4641-af30-bd0eec2be098 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:20.181330 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906cd4d-90e1-4242-ac1e-bc0053493762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:20.181985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906cd4d-90e1-4242-ac1e-bc0053493762 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:20.182247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906cd4d-90e1-4242-ac1e-bc0053493762 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:20.182519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906cd4d-90e1-4242-ac1e-bc0053493762 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:20.183129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906cd4d-90e1-4242-ac1e-bc0053493762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:20.184495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906cd4d-90e1-4242-ac1e-bc0053493762 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:20.197605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 444/896] 10.222.0.100 () {56 vars in 1092 bytes} [Thu Sep 1 17:24:20 2022] GET /identity/v3/services => generated 1274 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:24:20.197605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906cd4d-90e1-4242-ac1e-bc0053493762 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:20.305064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906cd4d-90e1-4242-ac1e-bc0053493762 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:20.309072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5f8d9-828b-4641-af30-bd0eec2be098 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:20.341963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:20.350392 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:20.351718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3dee0e-a7b0-47c5-ad73-b00dac92d1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:20.352737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dee0e-a7b0-47c5-ad73-b00dac92d1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:20.353233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dee0e-a7b0-47c5-ad73-b00dac92d1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:20.353757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dee0e-a7b0-47c5-ad73-b00dac92d1f5 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:20.355336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3dee0e-a7b0-47c5-ad73-b00dac92d1f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:20.355858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3dee0e-a7b0-47c5-ad73-b00dac92d1f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:20.357471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3dee0e-a7b0-47c5-ad73-b00dac92d1f5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:20.359738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94755b24-8648-4619-a47e-bc48e78b9704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:20.360819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94755b24-8648-4619-a47e-bc48e78b9704 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:20.361104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94755b24-8648-4619-a47e-bc48e78b9704 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:20.361383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94755b24-8648-4619-a47e-bc48e78b9704 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:20.364229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 445/897] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:20 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:20.374369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94755b24-8648-4619-a47e-bc48e78b9704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-94755b24-8648-4619-a47e-bc48e78b9704 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:20.380069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:20.381394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:20.382347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 453/898] 10.222.0.100 () {56 vars in 1125 bytes} [Thu Sep 1 17:24:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:20.388318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3dee0e-a7b0-47c5-ad73-b00dac92d1f5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:20.506461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3dee0e-a7b0-47c5-ad73-b00dac92d1f5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:20.543625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc414490-2fcc-4246-9531-ebcab5b24731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:20.544242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc414490-2fcc-4246-9531-ebcab5b24731 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:20.544520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc414490-2fcc-4246-9531-ebcab5b24731 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:20.544791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc414490-2fcc-4246-9531-ebcab5b24731 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:20.545455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc414490-2fcc-4246-9531-ebcab5b24731 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:20.545788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc414490-2fcc-4246-9531-ebcab5b24731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:20.547154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc414490-2fcc-4246-9531-ebcab5b24731 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:20.556135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 446/899] 10.222.0.100 () {56 vars in 1120 bytes} [Thu Sep 1 17:24:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:20.561736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94755b24-8648-4619-a47e-bc48e78b9704 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:20.680680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94755b24-8648-4619-a47e-bc48e78b9704 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:20.717594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e2e1-db83-4c42-a011-3eca0a6ef9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:20.718613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e2e1-db83-4c42-a011-3eca0a6ef9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:20.719211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e2e1-db83-4c42-a011-3eca0a6ef9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:20.719699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e2e1-db83-4c42-a011-3eca0a6ef9d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:20.731428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329e2e1-db83-4c42-a011-3eca0a6ef9d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:20.732894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329e2e1-db83-4c42-a011-3eca0a6ef9d7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3329e2e1-db83-4c42-a011-3eca0a6ef9d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:20.747742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:20.748772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:20.749759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:20.749759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:20.749759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:20.749759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 454/900] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:20.754639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc414490-2fcc-4246-9531-ebcab5b24731 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:20.819594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:20.866448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc414490-2fcc-4246-9531-ebcab5b24731 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:20.907908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11177c51-6f79-41cc-b6c7-d569c5cc18bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:20.908491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11177c51-6f79-41cc-b6c7-d569c5cc18bf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:20.908770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11177c51-6f79-41cc-b6c7-d569c5cc18bf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:20.909041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11177c51-6f79-41cc-b6c7-d569c5cc18bf None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:20.909768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11177c51-6f79-41cc-b6c7-d569c5cc18bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:20.910118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11177c51-6f79-41cc-b6c7-d569c5cc18bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:20.911485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11177c51-6f79-41cc-b6c7-d569c5cc18bf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:20.924685 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 447/901] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:20.930683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e2e1-db83-4c42-a011-3eca0a6ef9d7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:20.978581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:21.038885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e2e1-db83-4c42-a011-3eca0a6ef9d7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:21.077359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e64088-ea61-4b53-9a6f-cef04be539bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:21.078025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e64088-ea61-4b53-9a6f-cef04be539bd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:21.078303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e64088-ea61-4b53-9a6f-cef04be539bd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:21.078571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e64088-ea61-4b53-9a6f-cef04be539bd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:21.087159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e64088-ea61-4b53-9a6f-cef04be539bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:21.088591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e64088-ea61-4b53-9a6f-cef04be539bd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a1e64088-ea61-4b53-9a6f-cef04be539bd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:21.100808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:21.101892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:21.102843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 455/902] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:24:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:21.109805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11177c51-6f79-41cc-b6c7-d569c5cc18bf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:21.214625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11177c51-6f79-41cc-b6c7-d569c5cc18bf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:21.232698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:21.250805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5aa44-2b67-4381-a60e-95608b79b695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:21.251454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5aa44-2b67-4381-a60e-95608b79b695 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:21.251739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5aa44-2b67-4381-a60e-95608b79b695 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:21.252010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5aa44-2b67-4381-a60e-95608b79b695 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:21.252734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5aa44-2b67-4381-a60e-95608b79b695 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:21.253035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5aa44-2b67-4381-a60e-95608b79b695 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:21.254485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5aa44-2b67-4381-a60e-95608b79b695 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:21.274124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 448/903] 10.222.0.100 () {56 vars in 1154 bytes} [Thu Sep 1 17:24:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:21.280048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e64088-ea61-4b53-9a6f-cef04be539bd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:21.383983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:21.387407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e64088-ea61-4b53-9a6f-cef04be539bd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:21.425765 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58df84-10e4-4ce0-b7e6-c2a86305f048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:21.426380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58df84-10e4-4ce0-b7e6-c2a86305f048 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:21.426658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58df84-10e4-4ce0-b7e6-c2a86305f048 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:21.426928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58df84-10e4-4ce0-b7e6-c2a86305f048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:21.499569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58df84-10e4-4ce0-b7e6-c2a86305f048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:21.500950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58df84-10e4-4ce0-b7e6-c2a86305f048 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:21.602608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:21.607205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 456/904] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:21 2022] GET /identity/v3/auth/tokens => generated 1374 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:24:21.612908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5aa44-2b67-4381-a60e-95608b79b695 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:21.720241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5aa44-2b67-4381-a60e-95608b79b695 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:21.757267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a5e38-55d3-40a6-a369-fa67cd59d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:21.757852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a5e38-55d3-40a6-a369-fa67cd59d7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:21.758171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a5e38-55d3-40a6-a369-fa67cd59d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:21.758446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a5e38-55d3-40a6-a369-fa67cd59d7e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:21.758644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dade9-1da5-4d31-89cf-eca88f717a6a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:21.758925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dade9-1da5-4d31-89cf-eca88f717a6a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:21.759192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dade9-1da5-4d31-89cf-eca88f717a6a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:21.760750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 457/905] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:21.765755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49f943-43f2-4e29-87af-2d8919d659f8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:21.766039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49f943-43f2-4e29-87af-2d8919d659f8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:21.766314 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49f943-43f2-4e29-87af-2d8919d659f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:21.769262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a5e38-55d3-40a6-a369-fa67cd59d7e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:21.770760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a5e38-55d3-40a6-a369-fa67cd59d7e2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:21.785514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7e49f943-43f2-4e29-87af-2d8919d659f8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-363a5e38-55d3-40a6-a369-fa67cd59d7e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:21.786268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:21.787241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:21.788417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:21.788417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:21.788417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:21.788417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 449/906] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:21.789967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7e49f943-43f2-4e29-87af-2d8919d659f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:21.793287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a5e38-55d3-40a6-a369-fa67cd59d7e2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:21.822540 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:21.877868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 458/907] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:21 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:21.899888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a5e38-55d3-40a6-a369-fa67cd59d7e2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:21.936027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c124c2-c62b-4424-a030-03dd6de7ad4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:21.936563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c124c2-c62b-4424-a030-03dd6de7ad4e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:21.936849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c124c2-c62b-4424-a030-03dd6de7ad4e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:21.937100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c124c2-c62b-4424-a030-03dd6de7ad4e None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:21.937776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c124c2-c62b-4424-a030-03dd6de7ad4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:21.938077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c124c2-c62b-4424-a030-03dd6de7ad4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:21.939525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c124c2-c62b-4424-a030-03dd6de7ad4e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:21.952889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 450/908] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:21.958657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49f943-43f2-4e29-87af-2d8919d659f8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.062887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49f943-43f2-4e29-87af-2d8919d659f8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:22.096962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:22.104225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cc49c-e732-416d-b5dc-55411156bc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:22.104856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cc49c-e732-416d-b5dc-55411156bc31 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.105133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cc49c-e732-416d-b5dc-55411156bc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.105399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cc49c-e732-416d-b5dc-55411156bc31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.112867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cc49c-e732-416d-b5dc-55411156bc31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:22.114331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cc49c-e732-416d-b5dc-55411156bc31 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f93cc49c-e732-416d-b5dc-55411156bc31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:22.126889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:22.127850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:22.128752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:22.128752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:22.128752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:22.128752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:22.128752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:22.128752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:22.128752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:22.128752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:22.128752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 459/909] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:22.135654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c124c2-c62b-4424-a030-03dd6de7ad4e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.157597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cc49c-e732-416d-b5dc-55411156bc31 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.236586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:22.246281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c124c2-c62b-4424-a030-03dd6de7ad4e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:22.277410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cc49c-e732-416d-b5dc-55411156bc31 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:22.291828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44021b6-7fd5-4801-a147-2e1dda37d35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:22.292439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44021b6-7fd5-4801-a147-2e1dda37d35f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.292718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44021b6-7fd5-4801-a147-2e1dda37d35f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.293037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44021b6-7fd5-4801-a147-2e1dda37d35f None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.294139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44021b6-7fd5-4801-a147-2e1dda37d35f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:22.294445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44021b6-7fd5-4801-a147-2e1dda37d35f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:22.296182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44021b6-7fd5-4801-a147-2e1dda37d35f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:22.312008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 451/910] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:22.313693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6295daa-5ca5-41b1-a0e7-7e9f2fa9ceba None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.314200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6295daa-5ca5-41b1-a0e7-7e9f2fa9ceba None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.314546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6295daa-5ca5-41b1-a0e7-7e9f2fa9ceba None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.317627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 452/911] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:22.317627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6295daa-5ca5-41b1-a0e7-7e9f2fa9ceba None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.320928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c4a4-ef01-4532-a2ee-e2365d58cbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:22.321769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c4a4-ef01-4532-a2ee-e2365d58cbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.322213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c4a4-ef01-4532-a2ee-e2365d58cbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.322635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c4a4-ef01-4532-a2ee-e2365d58cbd0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.323373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c4a4-ef01-4532-a2ee-e2365d58cbd0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:22.324868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c4a4-ef01-4532-a2ee-e2365d58cbd0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-eb63c4a4-ef01-4532-a2ee-e2365d58cbd0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:22.335103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:22.336146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:22.337068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 460/912] 10.222.0.100 () {56 vars in 1137 bytes} [Thu Sep 1 17:24:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:22.337933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002f2b2-a5aa-4b73-83f6-f98eb0a20055 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.338365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002f2b2-a5aa-4b73-83f6-f98eb0a20055 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.338457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002f2b2-a5aa-4b73-83f6-f98eb0a20055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.357023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f002f2b2-a5aa-4b73-83f6-f98eb0a20055 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:22.358230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f002f2b2-a5aa-4b73-83f6-f98eb0a20055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:22.418147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:22.445805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 461/913] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:22 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:22.445805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002f2b2-a5aa-4b73-83f6-f98eb0a20055 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.450877 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6295daa-5ca5-41b1-a0e7-7e9f2fa9ceba None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:22.490600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e839344-3b88-4773-94b1-8c4a2ec9815a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:22.491413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e839344-3b88-4773-94b1-8c4a2ec9815a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.491693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e839344-3b88-4773-94b1-8c4a2ec9815a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.491973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e839344-3b88-4773-94b1-8c4a2ec9815a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.493329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e839344-3b88-4773-94b1-8c4a2ec9815a None admin] RBAC: Adding query filter params (role.id=42a25be79662491da6d022c8d6fb860f, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=71b9335f3f744a31a84ab278d9b45e36) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:22.493329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e839344-3b88-4773-94b1-8c4a2ec9815a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:22.494550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e839344-3b88-4773-94b1-8c4a2ec9815a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:22.509607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 453/914] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:22 2022] GET /identity/v3/role_assignments?user.id=71b9335f3f744a31a84ab278d9b45e36&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=42a25be79662491da6d022c8d6fb860f => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:22.535688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970729ea-c257-4d9a-913f-f91944aa6a1a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.536035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970729ea-c257-4d9a-913f-f91944aa6a1a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.536291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970729ea-c257-4d9a-913f-f91944aa6a1a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.539083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 454/915] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:22.544715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610435f1-d3d0-4d04-b523-6376a2b6d625 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.545017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610435f1-d3d0-4d04-b523-6376a2b6d625 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.545271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610435f1-d3d0-4d04-b523-6376a2b6d625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.566552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-610435f1-d3d0-4d04-b523-6376a2b6d625 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:22.567718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-610435f1-d3d0-4d04-b523-6376a2b6d625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:22.571883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:22.587805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002f2b2-a5aa-4b73-83f6-f98eb0a20055 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:22.629602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50e1c2-5c25-4b3e-b6c2-57987e08bfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:22.631709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50e1c2-5c25-4b3e-b6c2-57987e08bfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.632146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50e1c2-5c25-4b3e-b6c2-57987e08bfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.632577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50e1c2-5c25-4b3e-b6c2-57987e08bfc7 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.633428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50e1c2-5c25-4b3e-b6c2-57987e08bfc7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:22.633895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50e1c2-5c25-4b3e-b6c2-57987e08bfc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:22.635459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50e1c2-5c25-4b3e-b6c2-57987e08bfc7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:22.641801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 462/916] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:22 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:22.647812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50e1c2-5c25-4b3e-b6c2-57987e08bfc7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.688135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 455/917] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:22 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:22.722868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:22.724225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610435f1-d3d0-4d04-b523-6376a2b6d625 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.790980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50e1c2-5c25-4b3e-b6c2-57987e08bfc7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:22.829419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8407cb-bc8d-4259-881d-07fc7415d5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:22.830496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8407cb-bc8d-4259-881d-07fc7415d5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.830998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8407cb-bc8d-4259-881d-07fc7415d5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.831532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8407cb-bc8d-4259-881d-07fc7415d5f7 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.832386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8407cb-bc8d-4259-881d-07fc7415d5f7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:22.832905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8407cb-bc8d-4259-881d-07fc7415d5f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:22.835478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8407cb-bc8d-4259-881d-07fc7415d5f7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:22.841513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 463/918] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:22 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:22.847236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8407cb-bc8d-4259-881d-07fc7415d5f7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.857374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610435f1-d3d0-4d04-b523-6376a2b6d625 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:22.896406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26120d85-6799-4826-8f42-5869b9b2a053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:22.897785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26120d85-6799-4826-8f42-5869b9b2a053 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:22.898165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26120d85-6799-4826-8f42-5869b9b2a053 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:22.898537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26120d85-6799-4826-8f42-5869b9b2a053 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:22.899230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26120d85-6799-4826-8f42-5869b9b2a053 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:22.900702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26120d85-6799-4826-8f42-5869b9b2a053 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-26120d85-6799-4826-8f42-5869b9b2a053 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:22.911006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:22.912112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:24:22.912943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:22.917816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 456/919] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:24:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:22.919169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26120d85-6799-4826-8f42-5869b9b2a053 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:22.933962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:22.980929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8407cb-bc8d-4259-881d-07fc7415d5f7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:22.998141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:23.023288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27ad45-3a1a-47cf-a1e1-7de52123a63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:23.023984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27ad45-3a1a-47cf-a1e1-7de52123a63a None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:23.024166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27ad45-3a1a-47cf-a1e1-7de52123a63a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:23.024431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27ad45-3a1a-47cf-a1e1-7de52123a63a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:23.025091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27ad45-3a1a-47cf-a1e1-7de52123a63a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:23.026469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27ad45-3a1a-47cf-a1e1-7de52123a63a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:23.031291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26120d85-6799-4826-8f42-5869b9b2a053 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:23.052800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 464/920] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:24:22 2022] POST /identity/v3/endpoints => generated 349 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:24:23.054573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27ad45-3a1a-47cf-a1e1-7de52123a63a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:23.075116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bf89a-2e81-463c-a522-7e27cdb74942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:23.075845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bf89a-2e81-463c-a522-7e27cdb74942 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:23.076197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bf89a-2e81-463c-a522-7e27cdb74942 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:23.076544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bf89a-2e81-463c-a522-7e27cdb74942 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:23.077312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bf89a-2e81-463c-a522-7e27cdb74942 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:23.078111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bf89a-2e81-463c-a522-7e27cdb74942 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:23.079307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bf89a-2e81-463c-a522-7e27cdb74942 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:23.090810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 457/921] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:24:22 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:23.092338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bf89a-2e81-463c-a522-7e27cdb74942 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:23.183059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:23.206943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27ad45-3a1a-47cf-a1e1-7de52123a63a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:23.214459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bf89a-2e81-463c-a522-7e27cdb74942 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:23.256894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6d985-8459-4035-81bb-532f6e85a463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:23.257516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6d985-8459-4035-81bb-532f6e85a463 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:23.257863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6d985-8459-4035-81bb-532f6e85a463 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:23.258138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6d985-8459-4035-81bb-532f6e85a463 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:23.258794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6d985-8459-4035-81bb-532f6e85a463 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:23.260172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6d985-8459-4035-81bb-532f6e85a463 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:23.262662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259fbd5-b0de-48a3-a538-48030dacc00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:23.263439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259fbd5-b0de-48a3-a538-48030dacc00c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:23.263907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259fbd5-b0de-48a3-a538-48030dacc00c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:23.264377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259fbd5-b0de-48a3-a538-48030dacc00c None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:23.265022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259fbd5-b0de-48a3-a538-48030dacc00c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:23.265330 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259fbd5-b0de-48a3-a538-48030dacc00c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:23.266726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259fbd5-b0de-48a3-a538-48030dacc00c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:23.274650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 458/922] 10.222.0.100 () {56 vars in 1119 bytes} [Thu Sep 1 17:24:23 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:23.282100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259fbd5-b0de-48a3-a538-48030dacc00c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:23.283111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 465/923] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:24:23 2022] POST /identity/v3/services => generated 242 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:23.403300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259fbd5-b0de-48a3-a538-48030dacc00c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:23.434854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:23.449465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500ca51-1ddb-4491-b63a-ae81baa07bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:23.450106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500ca51-1ddb-4491-b63a-ae81baa07bb2 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:23.450386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500ca51-1ddb-4491-b63a-ae81baa07bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:23.450738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500ca51-1ddb-4491-b63a-ae81baa07bb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:23.451475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500ca51-1ddb-4491-b63a-ae81baa07bb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:23.452893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500ca51-1ddb-4491-b63a-ae81baa07bb2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:23.476269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 459/924] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:24:23 2022] POST /identity/v3/endpoints => generated 348 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:24.495330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7885b-2c91-4860-b2ef-617a620de2a6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:24.495855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7885b-2c91-4860-b2ef-617a620de2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:24.495855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7885b-2c91-4860-b2ef-617a620de2a6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:24.497437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 466/925] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:24.502929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ea1f0-3c55-43c0-8922-56deb82c1915 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:24.503349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ea1f0-3c55-43c0-8922-56deb82c1915 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:24.503493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ea1f0-3c55-43c0-8922-56deb82c1915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:24.526643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c37ea1f0-3c55-43c0-8922-56deb82c1915 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:24.528097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c37ea1f0-3c55-43c0-8922-56deb82c1915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:24.603587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 460/926] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:24 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:24.882983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7885b-2c91-4860-b2ef-617a620de2a6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:24.916048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:25.006842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7885b-2c91-4860-b2ef-617a620de2a6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:25.042538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82bc81-fbaf-4c4b-bcd7-eac6a58474c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:25.043179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82bc81-fbaf-4c4b-bcd7-eac6a58474c0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.043462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82bc81-fbaf-4c4b-bcd7-eac6a58474c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.043739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82bc81-fbaf-4c4b-bcd7-eac6a58474c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.055814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82bc81-fbaf-4c4b-bcd7-eac6a58474c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-ed82bc81-fbaf-4c4b-bcd7-eac6a58474c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:25.060663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:25.061633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:25.062497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 467/927] 10.222.0.100 () {56 vars in 1125 bytes} [Thu Sep 1 17:24:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:25.067832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ea1f0-3c55-43c0-8922-56deb82c1915 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:25.104915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299e763-902a-4f8b-ba59-63ff4185735f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.105270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299e763-902a-4f8b-ba59-63ff4185735f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.105482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299e763-902a-4f8b-ba59-63ff4185735f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.107005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 468/928] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:25.112312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147697b7-abdb-4f11-b775-038c73392084 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.112598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147697b7-abdb-4f11-b775-038c73392084 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.112873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147697b7-abdb-4f11-b775-038c73392084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.132211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-147697b7-abdb-4f11-b775-038c73392084 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:25.133495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-147697b7-abdb-4f11-b775-038c73392084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:25.176687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ea1f0-3c55-43c0-8922-56deb82c1915 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:25.188007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:25.206403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 469/929] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:25 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:25.226560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fe526-9ba9-4c9f-88e4-90d34d9ae33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:25.227571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fe526-9ba9-4c9f-88e4-90d34d9ae33e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.228072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fe526-9ba9-4c9f-88e4-90d34d9ae33e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.228594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fe526-9ba9-4c9f-88e4-90d34d9ae33e None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.229516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fe526-9ba9-4c9f-88e4-90d34d9ae33e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:25.230043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fe526-9ba9-4c9f-88e4-90d34d9ae33e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:25.231654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fe526-9ba9-4c9f-88e4-90d34d9ae33e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:25.245434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 461/930] 10.222.0.100 () {56 vars in 1120 bytes} [Thu Sep 1 17:24:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:25.247708 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147697b7-abdb-4f11-b775-038c73392084 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:25.274022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:25.352879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147697b7-abdb-4f11-b775-038c73392084 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:25.386206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1fd4c-937b-45ae-ad39-234af8c5d9d5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.386413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1fd4c-937b-45ae-ad39-234af8c5d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.386677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1fd4c-937b-45ae-ad39-234af8c5d9d5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.388502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 462/931] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:25.390401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad4024-e9c6-47df-a649-51cec2420d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:25.391306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad4024-e9c6-47df-a649-51cec2420d23 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.391802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad4024-e9c6-47df-a649-51cec2420d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.392338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad4024-e9c6-47df-a649-51cec2420d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.393528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d73d9-34a9-4dc2-bd92-99c05cee933c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.393837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d73d9-34a9-4dc2-bd92-99c05cee933c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.394109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d73d9-34a9-4dc2-bd92-99c05cee933c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.405542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad4024-e9c6-47df-a649-51cec2420d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:25.407473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad4024-e9c6-47df-a649-51cec2420d23 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:25.417006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bd9d73d9-34a9-4dc2-bd92-99c05cee933c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:25.418654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bd9d73d9-34a9-4dc2-bd92-99c05cee933c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f4ad4024-e9c6-47df-a649-51cec2420d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:25.424378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:25.425338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:25.426275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:25.426275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:25.426275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:25.428758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 470/932] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:25.435054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad4024-e9c6-47df-a649-51cec2420d23 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:25.498292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 463/933] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:25 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:25.499478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d73d9-34a9-4dc2-bd92-99c05cee933c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:25.524508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:25.567372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad4024-e9c6-47df-a649-51cec2420d23 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:25.607175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e68de-63e3-4fe7-988a-14378de24d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:25.607791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e68de-63e3-4fe7-988a-14378de24d8a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.608061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e68de-63e3-4fe7-988a-14378de24d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.608340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e68de-63e3-4fe7-988a-14378de24d8a None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.608998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e68de-63e3-4fe7-988a-14378de24d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:25.609294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e68de-63e3-4fe7-988a-14378de24d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:25.610692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e68de-63e3-4fe7-988a-14378de24d8a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:25.611082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:25.615520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d73d9-34a9-4dc2-bd92-99c05cee933c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:25.630109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 471/934] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:25.630109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247b3bb-ec52-4253-a892-24b4da4d1284 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.630109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247b3bb-ec52-4253-a892-24b4da4d1284 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.630109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247b3bb-ec52-4253-a892-24b4da4d1284 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.630109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 472/935] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:25.635551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275b1e1-a0dd-4909-b49a-2d46c33a7961 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.635832 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275b1e1-a0dd-4909-b49a-2d46c33a7961 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.636108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275b1e1-a0dd-4909-b49a-2d46c33a7961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.651387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0af07-3447-49e8-bbcc-0f4badc51fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:25.652021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0af07-3447-49e8-bbcc-0f4badc51fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.652295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0af07-3447-49e8-bbcc-0f4badc51fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.652563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0af07-3447-49e8-bbcc-0f4badc51fa1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.653393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0af07-3447-49e8-bbcc-0f4badc51fa1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:25.654800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0af07-3447-49e8-bbcc-0f4badc51fa1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:25.660295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5275b1e1-a0dd-4909-b49a-2d46c33a7961 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:25.661711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5275b1e1-a0dd-4909-b49a-2d46c33a7961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-dda0af07-3447-49e8-bbcc-0f4badc51fa1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:25.667308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:25.668395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:25.669888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 464/936] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:25.670707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0af07-3447-49e8-bbcc-0f4badc51fa1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:25.734225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 473/937] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:25 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:25.735412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275b1e1-a0dd-4909-b49a-2d46c33a7961 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:25.803975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0af07-3447-49e8-bbcc-0f4badc51fa1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:25.808189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:25.842121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab0836-f2d3-4fc5-8ae9-6d4bcbeeb69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:25.842747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab0836-f2d3-4fc5-8ae9-6d4bcbeeb69e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.843104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab0836-f2d3-4fc5-8ae9-6d4bcbeeb69e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.843304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab0836-f2d3-4fc5-8ae9-6d4bcbeeb69e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.854558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab0836-f2d3-4fc5-8ae9-6d4bcbeeb69e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:25.857170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab0836-f2d3-4fc5-8ae9-6d4bcbeeb69e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:25.859986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275b1e1-a0dd-4909-b49a-2d46c33a7961 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-abab0836-f2d3-4fc5-8ae9-6d4bcbeeb69e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:25.872396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:25.873425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:25.874416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:25.877888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 465/938] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:24:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:25.878972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab0836-f2d3-4fc5-8ae9-6d4bcbeeb69e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:25.904833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:25.905935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce06bf2-b57e-48cb-89b9-9cd56e49ffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:25.906869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce06bf2-b57e-48cb-89b9-9cd56e49ffa5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:25.907466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce06bf2-b57e-48cb-89b9-9cd56e49ffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:25.907893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce06bf2-b57e-48cb-89b9-9cd56e49ffa5 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:25.908674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce06bf2-b57e-48cb-89b9-9cd56e49ffa5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:25.909116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce06bf2-b57e-48cb-89b9-9cd56e49ffa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:25.910682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce06bf2-b57e-48cb-89b9-9cd56e49ffa5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:25.916648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 474/939] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:25 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:25.917651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce06bf2-b57e-48cb-89b9-9cd56e49ffa5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:25.992732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab0836-f2d3-4fc5-8ae9-6d4bcbeeb69e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.028951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53f3fa-0e07-44e0-8aa3-c50de561209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.029604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53f3fa-0e07-44e0-8aa3-c50de561209f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.029882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53f3fa-0e07-44e0-8aa3-c50de561209f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.030155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53f3fa-0e07-44e0-8aa3-c50de561209f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.030658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce06bf2-b57e-48cb-89b9-9cd56e49ffa5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.030755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53f3fa-0e07-44e0-8aa3-c50de561209f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.032088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53f3fa-0e07-44e0-8aa3-c50de561209f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1a53f3fa-0e07-44e0-8aa3-c50de561209f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:26.040901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:26.041984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:26.043052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 466/940] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:26.044654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53f3fa-0e07-44e0-8aa3-c50de561209f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.056979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:26.069077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d4289-d36c-4718-95b1-1da515a80f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.069811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d4289-d36c-4718-95b1-1da515a80f0b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.070094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d4289-d36c-4718-95b1-1da515a80f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.070364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d4289-d36c-4718-95b1-1da515a80f0b None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.071094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d4289-d36c-4718-95b1-1da515a80f0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:26.071399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d4289-d36c-4718-95b1-1da515a80f0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.072828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d4289-d36c-4718-95b1-1da515a80f0b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.094863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 475/941] 10.222.0.100 () {56 vars in 1154 bytes} [Thu Sep 1 17:24:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:26.094863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d4289-d36c-4718-95b1-1da515a80f0b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.132361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:26.147866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53f3fa-0e07-44e0-8aa3-c50de561209f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.182285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a8df7-9a8a-4086-93bb-83a7dc979aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.182807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a8df7-9a8a-4086-93bb-83a7dc979aab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.183082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a8df7-9a8a-4086-93bb-83a7dc979aab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.183346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a8df7-9a8a-4086-93bb-83a7dc979aab None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.183960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a8df7-9a8a-4086-93bb-83a7dc979aab None admin] RBAC: Adding query filter params (type=compute) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:26.184254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a8df7-9a8a-4086-93bb-83a7dc979aab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.185650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a8df7-9a8a-4086-93bb-83a7dc979aab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.189959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 467/942] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:26.190721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a8df7-9a8a-4086-93bb-83a7dc979aab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.191179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d4289-d36c-4718-95b1-1da515a80f0b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.237088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.237743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.238022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.238296 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.238960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.240983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.243776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WARNING keystone.server.flask.common [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 17:24:26.263821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.sql.core [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 17:24:26.263821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 17:24:26.263821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [parameters: {'id': '298f2dfcc4e4491582a2a6fbbc56173c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 17:24:26.263821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81532) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 298f2dfcc4e4491582a2a6fbbc56173c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 298f2dfcc4e4491582a2a6fbbc56173c. Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:24:26.271500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 17:24:26.272748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application yield resource Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise self.value Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application session.commit() Sep 01 17:24:26.274112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.flush() Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application compat.raise_( Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 17:24:26.275048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 17:24:26.275956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:24:26.276746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [parameters: {'id': '298f2dfcc4e4491582a2a6fbbc56173c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:26.277653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:26.279281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 298f2dfcc4e4491582a2a6fbbc56173c. Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 476/943] 10.222.0.100 () {60 vars in 1145 bytes} [Thu Sep 1 17:24:26 2022] POST /identity/v3/projects => generated 262 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:26.280616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.323731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:26.324827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a8df7-9a8a-4086-93bb-83a7dc979aab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.366748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f125eaa-8c41-4dea-982b-a682195d7097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.367456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f125eaa-8c41-4dea-982b-a682195d7097 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.367739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f125eaa-8c41-4dea-982b-a682195d7097 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.368007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f125eaa-8c41-4dea-982b-a682195d7097 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.368664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f125eaa-8c41-4dea-982b-a682195d7097 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:26.368969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f125eaa-8c41-4dea-982b-a682195d7097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.370361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f125eaa-8c41-4dea-982b-a682195d7097 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.375500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 468/944] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:26.376075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f125eaa-8c41-4dea-982b-a682195d7097 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.391731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:26.407991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd076b-946e-4326-8762-b1437ea1904c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.447244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171a995-69fe-4cb3-91cc-3295f25a9a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.447792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171a995-69fe-4cb3-91cc-3295f25a9a9a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.448068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171a995-69fe-4cb3-91cc-3295f25a9a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.448334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171a995-69fe-4cb3-91cc-3295f25a9a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.460344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171a995-69fe-4cb3-91cc-3295f25a9a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.462338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171a995-69fe-4cb3-91cc-3295f25a9a9a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4171a995-69fe-4cb3-91cc-3295f25a9a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:26.478434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:26.479446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:26.480340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:26.480340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:26.480340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:26.480340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 477/945] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:26.481051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171a995-69fe-4cb3-91cc-3295f25a9a9a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.490854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f125eaa-8c41-4dea-982b-a682195d7097 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.528873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9337c37-7c65-4c3d-894a-22534ec01554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.529499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9337c37-7c65-4c3d-894a-22534ec01554 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.529802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9337c37-7c65-4c3d-894a-22534ec01554 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.530191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9337c37-7c65-4c3d-894a-22534ec01554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.538105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9337c37-7c65-4c3d-894a-22534ec01554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.539655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9337c37-7c65-4c3d-894a-22534ec01554 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e9337c37-7c65-4c3d-894a-22534ec01554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:26.550936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:26.551827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:26.552809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:26.552809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:26.552809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:26.552809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:26.552809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:26.552809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:26.552809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:26.552809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:26.552809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 469/946] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:26.553934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9337c37-7c65-4c3d-894a-22534ec01554 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.599588 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:26.611728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171a995-69fe-4cb3-91cc-3295f25a9a9a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.636329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:26.655346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7acc7-00c3-47bb-aac5-919e5eb76212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.656105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7acc7-00c3-47bb-aac5-919e5eb76212 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.656344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7acc7-00c3-47bb-aac5-919e5eb76212 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.656509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7acc7-00c3-47bb-aac5-919e5eb76212 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.657146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7acc7-00c3-47bb-aac5-919e5eb76212 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:26.657615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7acc7-00c3-47bb-aac5-919e5eb76212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.659635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7acc7-00c3-47bb-aac5-919e5eb76212 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.668420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9337c37-7c65-4c3d-894a-22534ec01554 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.669826 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 478/947] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:26.670473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7acc7-00c3-47bb-aac5-919e5eb76212 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.712238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca31f1-3967-42df-9b14-106a1e00ad28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.712773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca31f1-3967-42df-9b14-106a1e00ad28 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.713054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca31f1-3967-42df-9b14-106a1e00ad28 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.713329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca31f1-3967-42df-9b14-106a1e00ad28 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.714010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca31f1-3967-42df-9b14-106a1e00ad28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:26.714315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca31f1-3967-42df-9b14-106a1e00ad28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.715691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca31f1-3967-42df-9b14-106a1e00ad28 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.728494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 470/948] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:26.734147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca31f1-3967-42df-9b14-106a1e00ad28 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.783767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7acc7-00c3-47bb-aac5-919e5eb76212 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.836420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309ad53-fa13-4268-83e3-7c9e1b0001fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.837378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309ad53-fa13-4268-83e3-7c9e1b0001fb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.837760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309ad53-fa13-4268-83e3-7c9e1b0001fb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.838036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309ad53-fa13-4268-83e3-7c9e1b0001fb None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.838704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309ad53-fa13-4268-83e3-7c9e1b0001fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:26.839005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309ad53-fa13-4268-83e3-7c9e1b0001fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.840433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309ad53-fa13-4268-83e3-7c9e1b0001fb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.860240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca31f1-3967-42df-9b14-106a1e00ad28 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:26.873840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 479/949] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:26.873840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309ad53-fa13-4268-83e3-7c9e1b0001fb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:26.889916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:26.909083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b51ddf-e4d6-4342-9255-e9e073ac4eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:26.909729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b51ddf-e4d6-4342-9255-e9e073ac4eab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:26.910011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b51ddf-e4d6-4342-9255-e9e073ac4eab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:26.910282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b51ddf-e4d6-4342-9255-e9e073ac4eab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:26.911048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b51ddf-e4d6-4342-9255-e9e073ac4eab None admin] RBAC: Adding query filter params (interface=public, service_id=84bd851758614e7f8678ae6eb88f65dc, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:26.911358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b51ddf-e4d6-4342-9255-e9e073ac4eab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:26.913197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b51ddf-e4d6-4342-9255-e9e073ac4eab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:26.915135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:26.920331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 471/950] 10.222.0.100 () {56 vars in 1256 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/endpoints?service_id=84bd851758614e7f8678ae6eb88f65dc&interface=public®ion_id=RegionOne => generated 195 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:26.925810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b51ddf-e4d6-4342-9255-e9e073ac4eab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:27.009391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309ad53-fa13-4268-83e3-7c9e1b0001fb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:27.047109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b51ddf-e4d6-4342-9255-e9e073ac4eab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:27.056959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9172c-3a24-44cb-8d10-37b23d39cbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:27.058067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9172c-3a24-44cb-8d10-37b23d39cbda None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:27.058067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9172c-3a24-44cb-8d10-37b23d39cbda None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:27.058280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9172c-3a24-44cb-8d10-37b23d39cbda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:27.066062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9172c-3a24-44cb-8d10-37b23d39cbda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:27.068452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9172c-3a24-44cb-8d10-37b23d39cbda None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6bd9172c-3a24-44cb-8d10-37b23d39cbda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:27.079976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:27.081025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:27.082802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:27.082802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:27.082802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:27.082802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:27.082802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:27.082802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:27.082802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:27.082802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:27.082802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 480/951] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:27.087487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9172c-3a24-44cb-8d10-37b23d39cbda None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:27.091504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd198cc2-b658-4e08-99b3-72badc1bbadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:27.092117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd198cc2-b658-4e08-99b3-72badc1bbadb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:27.092398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd198cc2-b658-4e08-99b3-72badc1bbadb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:27.092670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd198cc2-b658-4e08-99b3-72badc1bbadb None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:27.093279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd198cc2-b658-4e08-99b3-72badc1bbadb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:27.094706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd198cc2-b658-4e08-99b3-72badc1bbadb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:27.105664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 472/952] 10.222.0.100 () {56 vars in 1092 bytes} [Thu Sep 1 17:24:26 2022] GET /identity/v3/services => generated 1504 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:24:27.186346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:27.217606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9172c-3a24-44cb-8d10-37b23d39cbda None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:27.262292 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b153961-0821-4b52-a2a5-59661cb045fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:27.263212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b153961-0821-4b52-a2a5-59661cb045fe None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:27.263670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b153961-0821-4b52-a2a5-59661cb045fe None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:27.263946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b153961-0821-4b52-a2a5-59661cb045fe None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:27.264647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b153961-0821-4b52-a2a5-59661cb045fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:27.264952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b153961-0821-4b52-a2a5-59661cb045fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:27.266414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b153961-0821-4b52-a2a5-59661cb045fe None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:27.281474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 481/953] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:27.288216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd198cc2-b658-4e08-99b3-72badc1bbadb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:27.389969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:27.439241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd198cc2-b658-4e08-99b3-72badc1bbadb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:27.477475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a93ae-718c-4d80-a7e8-264918b51216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:27.478329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a93ae-718c-4d80-a7e8-264918b51216 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:27.478743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a93ae-718c-4d80-a7e8-264918b51216 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:27.479065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a93ae-718c-4d80-a7e8-264918b51216 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/71b9335f3f744a31a84ab278d9b45e36/roles/42a25be79662491da6d022c8d6fb860f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:27.481424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a93ae-718c-4d80-a7e8-264918b51216 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=71b9335f3f744a31a84ab278d9b45e36, role_id=42a25be79662491da6d022c8d6fb860f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:27.482959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a93ae-718c-4d80-a7e8-264918b51216 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:27.499956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 473/954] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:24:27 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/71b9335f3f744a31a84ab278d9b45e36/roles/42a25be79662491da6d022c8d6fb860f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:24:28.217808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a0951-8a8c-4ea6-ab1a-fe81cf336d41 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:28.217808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a0951-8a8c-4ea6-ab1a-fe81cf336d41 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:28.217808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a0951-8a8c-4ea6-ab1a-fe81cf336d41 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:28.218978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 482/955] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:28 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:28.224519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6154f9f-c530-4292-a8a6-f416830c5ed1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:28.225318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6154f9f-c530-4292-a8a6-f416830c5ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:28.225820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6154f9f-c530-4292-a8a6-f416830c5ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:28.248000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d6154f9f-c530-4292-a8a6-f416830c5ed1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:28.249507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d6154f9f-c530-4292-a8a6-f416830c5ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:28.322357 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 474/956] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:28 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:28.593380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a0951-8a8c-4ea6-ab1a-fe81cf336d41 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:28.702347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a0951-8a8c-4ea6-ab1a-fe81cf336d41 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:28.738782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41314f37-30c7-468d-b8e0-6b7d8d92b5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:28.739396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41314f37-30c7-468d-b8e0-6b7d8d92b5b5 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:28.739672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41314f37-30c7-468d-b8e0-6b7d8d92b5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:28.739963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41314f37-30c7-468d-b8e0-6b7d8d92b5b5 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:28.741161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41314f37-30c7-468d-b8e0-6b7d8d92b5b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:28.742715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41314f37-30c7-468d-b8e0-6b7d8d92b5b5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:28.746174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:28.761452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 483/957] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:24:28 2022] POST /identity/v3/services => generated 243 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:29.163019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0f68a-eaa0-4280-a191-60c8e4f9f8fc None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:29.163728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0f68a-eaa0-4280-a191-60c8e4f9f8fc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:29.164219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0f68a-eaa0-4280-a191-60c8e4f9f8fc None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:29.166141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 475/958] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:29 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:29.172193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3414434-8ef9-45c5-b914-489a3bb748c1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:29.172758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3414434-8ef9-45c5-b914-489a3bb748c1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:29.173227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3414434-8ef9-45c5-b914-489a3bb748c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:29.186729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6ce4-dfde-4285-9ab5-9144ee359db0 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:29.187039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6ce4-dfde-4285-9ab5-9144ee359db0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:29.187283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6ce4-dfde-4285-9ab5-9144ee359db0 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:29.188795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 476/959] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:29.195056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a0c6f-1aeb-4076-ac47-41d7ae31153d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:29.195361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a0c6f-1aeb-4076-ac47-41d7ae31153d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:29.195637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a0c6f-1aeb-4076-ac47-41d7ae31153d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:29.196648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e3414434-8ef9-45c5-b914-489a3bb748c1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:29.198442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e3414434-8ef9-45c5-b914-489a3bb748c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:29.215547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f24a0c6f-1aeb-4076-ac47-41d7ae31153d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:29.216751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f24a0c6f-1aeb-4076-ac47-41d7ae31153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:29.277600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:29.300053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 484/960] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:29 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:29.300629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 477/961] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:29 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:29.572744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3414434-8ef9-45c5-b914-489a3bb748c1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:29.575398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a0c6f-1aeb-4076-ac47-41d7ae31153d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:29.695835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3414434-8ef9-45c5-b914-489a3bb748c1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:29.705333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a0c6f-1aeb-4076-ac47-41d7ae31153d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:29.720658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:29.744285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083ee76-9782-4be0-937b-a03fe6795e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:29.744928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083ee76-9782-4be0-937b-a03fe6795e0f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:29.745226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083ee76-9782-4be0-937b-a03fe6795e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:29.745476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083ee76-9782-4be0-937b-a03fe6795e0f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:29.746150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083ee76-9782-4be0-937b-a03fe6795e0f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:29.747467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083ee76-9782-4be0-937b-a03fe6795e0f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:29.750981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f47d8f-02e8-4a1b-8281-ec3ef224a6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:29.751659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f47d8f-02e8-4a1b-8281-ec3ef224a6ce None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:29.752007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f47d8f-02e8-4a1b-8281-ec3ef224a6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:29.752351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f47d8f-02e8-4a1b-8281-ec3ef224a6ce None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:29.753070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f47d8f-02e8-4a1b-8281-ec3ef224a6ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:29.755235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f47d8f-02e8-4a1b-8281-ec3ef224a6ce None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b083ee76-9782-4be0-937b-a03fe6795e0f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:29.757744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:29.758850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:29.759755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:29.760525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING keystone.server.flask.common [None req-11f47d8f-02e8-4a1b-8281-ec3ef224a6ce None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 17:24:29.763710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 485/962] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:29.765039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a922916-9b4b-4059-b7d3-3c8c13b42477 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:29.765324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a922916-9b4b-4059-b7d3-3c8c13b42477 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:29.765611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a922916-9b4b-4059-b7d3-3c8c13b42477 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:29.766998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 486/963] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:29.767614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a922916-9b4b-4059-b7d3-3c8c13b42477 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:29.779382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.sql.core [None req-11f47d8f-02e8-4a1b-8281-ec3ef224a6ce None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 17:24:29.779382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 17:24:29.779382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [parameters: {'user_id': '087690ad1e574d4f8b8992fe05a936c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 17:24:29.779382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81533) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-11f47d8f-02e8-4a1b-8281-ec3ef224a6ce None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result.read() Sep 01 17:24:29.796195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:24:29.797170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application yield resource Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise self.value Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application session.commit() Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 17:24:29.798210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.session.flush() Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application compat.raise_( Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 17:24:29.799127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:24:29.800065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result.read() Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:24:29.800948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [parameters: {'user_id': '087690ad1e574d4f8b8992fe05a936c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:29.803874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 17:24:29.804695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 478/964] 10.222.0.100 () {60 vars in 1136 bytes} [Thu Sep 1 17:24:29 2022] POST /identity/v3/users => generated 160 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d375a-68f0-42c8-8805-675e640ae844 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d375a-68f0-42c8-8805-675e640ae844 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:29.805626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d375a-68f0-42c8-8805-675e640ae844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:29.822066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-434d375a-68f0-42c8-8805-675e640ae844 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:29.823702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-434d375a-68f0-42c8-8805-675e640ae844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:29.872579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:29.899944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a922916-9b4b-4059-b7d3-3c8c13b42477 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:29.907045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 479/965] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:29 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:29.907761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d375a-68f0-42c8-8805-675e640ae844 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:29.960998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac2404-4c49-48ba-8f48-368a4f14972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:29.962267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac2404-4c49-48ba-8f48-368a4f14972e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:29.962702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac2404-4c49-48ba-8f48-368a4f14972e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:29.963397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac2404-4c49-48ba-8f48-368a4f14972e None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:29.964768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac2404-4c49-48ba-8f48-368a4f14972e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:29.965255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac2404-4c49-48ba-8f48-368a4f14972e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:29.966878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac2404-4c49-48ba-8f48-368a4f14972e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:29.974750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 487/966] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:29 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:29.982009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac2404-4c49-48ba-8f48-368a4f14972e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:30.039689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d375a-68f0-42c8-8805-675e640ae844 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:30.043827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:30.079905 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07d3e2-add5-497d-b61b-6bf824d7345e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:30.080464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07d3e2-add5-497d-b61b-6bf824d7345e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.080747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07d3e2-add5-497d-b61b-6bf824d7345e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.081011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07d3e2-add5-497d-b61b-6bf824d7345e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.089368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07d3e2-add5-497d-b61b-6bf824d7345e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:30.091074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07d3e2-add5-497d-b61b-6bf824d7345e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1a07d3e2-add5-497d-b61b-6bf824d7345e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:30.105625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:30.106627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:24:30.107651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:30.111705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 480/967] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:24:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:30.113019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07d3e2-add5-497d-b61b-6bf824d7345e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:30.123792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac2404-4c49-48ba-8f48-368a4f14972e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:30.154015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:30.173920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5a9a-e181-445a-bf52-1e56616864d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:30.174876 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5a9a-e181-445a-bf52-1e56616864d7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.175875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5a9a-e181-445a-bf52-1e56616864d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.176334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5a9a-e181-445a-bf52-1e56616864d7 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.178000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a5a9a-e181-445a-bf52-1e56616864d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:30.178489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a5a9a-e181-445a-bf52-1e56616864d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:30.180787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a5a9a-e181-445a-bf52-1e56616864d7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:30.189046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 488/968] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:29 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:30.190243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5a9a-e181-445a-bf52-1e56616864d7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:30.230368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07d3e2-add5-497d-b61b-6bf824d7345e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:30.273369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b25dda-bd76-4a8b-8b24-970f7ca49469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:30.274342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b25dda-bd76-4a8b-8b24-970f7ca49469 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.274822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b25dda-bd76-4a8b-8b24-970f7ca49469 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.275097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b25dda-bd76-4a8b-8b24-970f7ca49469 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.275804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b25dda-bd76-4a8b-8b24-970f7ca49469 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:30.276108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b25dda-bd76-4a8b-8b24-970f7ca49469 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:30.277540 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b25dda-bd76-4a8b-8b24-970f7ca49469 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:30.315747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5a9a-e181-445a-bf52-1e56616864d7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:30.333783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 481/969] 10.222.0.100 () {56 vars in 1106 bytes} [Thu Sep 1 17:24:30 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:30.333783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b25dda-bd76-4a8b-8b24-970f7ca49469 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:30.335513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:30.368014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d367f5-a061-4cb3-bffc-37d027042ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:30.369058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d367f5-a061-4cb3-bffc-37d027042ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.369416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d367f5-a061-4cb3-bffc-37d027042ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.369807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d367f5-a061-4cb3-bffc-37d027042ec1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.383897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d367f5-a061-4cb3-bffc-37d027042ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-15d367f5-a061-4cb3-bffc-37d027042ec1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:30.389701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:30.390914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:30.392012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 489/970] 10.222.0.100 () {56 vars in 1125 bytes} [Thu Sep 1 17:24:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:30.400304 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d367f5-a061-4cb3-bffc-37d027042ec1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:30.455387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:30.464717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b25dda-bd76-4a8b-8b24-970f7ca49469 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:30.513685 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8ae4b-3ab1-4df0-93a6-84d02fb8b96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:30.514319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ae4b-3ab1-4df0-93a6-84d02fb8b96f None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.514597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ae4b-3ab1-4df0-93a6-84d02fb8b96f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.514871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ae4b-3ab1-4df0-93a6-84d02fb8b96f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.515582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8ae4b-3ab1-4df0-93a6-84d02fb8b96f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:30.518355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8ae4b-3ab1-4df0-93a6-84d02fb8b96f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:30.545622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 482/971] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:24:30 2022] POST /identity/v3/endpoints => generated 351 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:30.548278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d367f5-a061-4cb3-bffc-37d027042ec1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:30.606947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59224ff1-8602-42dc-ae42-b0b4bc73856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:30.607716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224ff1-8602-42dc-ae42-b0b4bc73856d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.608005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224ff1-8602-42dc-ae42-b0b4bc73856d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.608272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224ff1-8602-42dc-ae42-b0b4bc73856d None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.608965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59224ff1-8602-42dc-ae42-b0b4bc73856d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:30.609274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59224ff1-8602-42dc-ae42-b0b4bc73856d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:30.612800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59224ff1-8602-42dc-ae42-b0b4bc73856d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:30.624085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 490/972] 10.222.0.100 () {56 vars in 1120 bytes} [Thu Sep 1 17:24:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:30.629995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8ae4b-3ab1-4df0-93a6-84d02fb8b96f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:30.752611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8ae4b-3ab1-4df0-93a6-84d02fb8b96f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:30.788763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf4bcf-9028-4477-847d-0c8ce787801f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:30.789397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4bcf-9028-4477-847d-0c8ce787801f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.789708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4bcf-9028-4477-847d-0c8ce787801f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.789983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4bcf-9028-4477-847d-0c8ce787801f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.800783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf4bcf-9028-4477-847d-0c8ce787801f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:30.802259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf4bcf-9028-4477-847d-0c8ce787801f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:30.811352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-eccf4bcf-9028-4477-847d-0c8ce787801f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:30.819762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:30.821005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:30.822128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:30.822128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:30.822128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:30.824248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 483/973] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:30.830751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59224ff1-8602-42dc-ae42-b0b4bc73856d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:30.844966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a824b2-4447-48c2-bb73-19c641b21e6b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.845260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a824b2-4447-48c2-bb73-19c641b21e6b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.845542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a824b2-4447-48c2-bb73-19c641b21e6b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.849611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 484/974] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:30.857407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf11000-cc51-49a4-a208-d257531f12db None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.857689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf11000-cc51-49a4-a208-d257531f12db None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.857978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf11000-cc51-49a4-a208-d257531f12db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.868384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:30.877472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dbf11000-cc51-49a4-a208-d257531f12db None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:30.878712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dbf11000-cc51-49a4-a208-d257531f12db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:30.952313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59224ff1-8602-42dc-ae42-b0b4bc73856d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:30.956840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 485/975] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:30 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:30.990779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ade6bc-4070-4a67-a577-4b4793898e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:30.991777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade6bc-4070-4a67-a577-4b4793898e30 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:30.992260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade6bc-4070-4a67-a577-4b4793898e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:30.992740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade6bc-4070-4a67-a577-4b4793898e30 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:30.993596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ade6bc-4070-4a67-a577-4b4793898e30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:30.994118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ade6bc-4070-4a67-a577-4b4793898e30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:30.995680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ade6bc-4070-4a67-a577-4b4793898e30 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:31.008885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 491/976] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:31.014811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf11000-cc51-49a4-a208-d257531f12db None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:31.128391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf11000-cc51-49a4-a208-d257531f12db None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:31.159755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:31.169100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5087b-c0b1-4143-b2c9-fb678e6e53a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:31.169777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5087b-c0b1-4143-b2c9-fb678e6e53a2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:31.170061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5087b-c0b1-4143-b2c9-fb678e6e53a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:31.170332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5087b-c0b1-4143-b2c9-fb678e6e53a2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:31.179050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5087b-c0b1-4143-b2c9-fb678e6e53a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:31.180499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5087b-c0b1-4143-b2c9-fb678e6e53a2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-26e5087b-c0b1-4143-b2c9-fb678e6e53a2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:31.192794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:31.194780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:24:31.196065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:31.197995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 486/977] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:24:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:31.200404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ade6bc-4070-4a67-a577-4b4793898e30 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:31.232448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5087b-c0b1-4143-b2c9-fb678e6e53a2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:31.306968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:31.319966 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ade6bc-4070-4a67-a577-4b4793898e30 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:31.349411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5087b-c0b1-4143-b2c9-fb678e6e53a2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:31.366580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259d495-d23c-4d27-82c2-9e7db70acd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:31.367558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259d495-d23c-4d27-82c2-9e7db70acd28 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:31.368016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259d495-d23c-4d27-82c2-9e7db70acd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:31.368499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259d495-d23c-4d27-82c2-9e7db70acd28 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:31.369880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259d495-d23c-4d27-82c2-9e7db70acd28 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:31.370429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259d495-d23c-4d27-82c2-9e7db70acd28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:31.373138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259d495-d23c-4d27-82c2-9e7db70acd28 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:31.387077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73a4d1-b2f9-452e-9008-8cc61009a7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:31.387628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73a4d1-b2f9-452e-9008-8cc61009a7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:31.387911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73a4d1-b2f9-452e-9008-8cc61009a7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:31.388195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73a4d1-b2f9-452e-9008-8cc61009a7f1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:31.388777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73a4d1-b2f9-452e-9008-8cc61009a7f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:31.390163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73a4d1-b2f9-452e-9008-8cc61009a7f1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-ae73a4d1-b2f9-452e-9008-8cc61009a7f1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:31.399823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:31.401716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:31.403586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 487/978] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:31.407859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73a4d1-b2f9-452e-9008-8cc61009a7f1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:31.412159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 492/979] 10.222.0.100 () {56 vars in 1154 bytes} [Thu Sep 1 17:24:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:31.418109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259d495-d23c-4d27-82c2-9e7db70acd28 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:31.488886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:31.537962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259d495-d23c-4d27-82c2-9e7db70acd28 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:31.543347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73a4d1-b2f9-452e-9008-8cc61009a7f1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:31.579009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b3a98-dbdd-4a17-9973-ecdc13842631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:31.579621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3a98-dbdd-4a17-9973-ecdc13842631 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:31.579900 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3a98-dbdd-4a17-9973-ecdc13842631 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:31.580172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3a98-dbdd-4a17-9973-ecdc13842631 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:31.584743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf06ab-eba3-43db-b0bb-bb8a5b468293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:31.585354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf06ab-eba3-43db-b0bb-bb8a5b468293 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:31.585661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf06ab-eba3-43db-b0bb-bb8a5b468293 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:31.585942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf06ab-eba3-43db-b0bb-bb8a5b468293 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:31.586714 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf06ab-eba3-43db-b0bb-bb8a5b468293 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:31.586909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf06ab-eba3-43db-b0bb-bb8a5b468293 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:31.588288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf06ab-eba3-43db-b0bb-bb8a5b468293 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:31.595805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 488/980] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:31 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:31.602351 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:31.602981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf06ab-eba3-43db-b0bb-bb8a5b468293 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:31.679811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b3a98-dbdd-4a17-9973-ecdc13842631 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:31.681267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b3a98-dbdd-4a17-9973-ecdc13842631 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:31.729247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf06ab-eba3-43db-b0bb-bb8a5b468293 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:31.748781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:31.768378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a552d-6a91-4c62-8b46-11bd04904f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:31.768994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a552d-6a91-4c62-8b46-11bd04904f8c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:31.769274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a552d-6a91-4c62-8b46-11bd04904f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:31.769591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a552d-6a91-4c62-8b46-11bd04904f8c None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:31.770249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a552d-6a91-4c62-8b46-11bd04904f8c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:31.770551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a552d-6a91-4c62-8b46-11bd04904f8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:31.772364 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a552d-6a91-4c62-8b46-11bd04904f8c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:31.777482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 489/981] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:31 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:31.783449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a552d-6a91-4c62-8b46-11bd04904f8c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:31.811298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 493/982] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:31 2022] GET /identity/v3/auth/tokens => generated 1572 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:24:31.817626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b3a98-dbdd-4a17-9973-ecdc13842631 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:31.875327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:31.928514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a552d-6a91-4c62-8b46-11bd04904f8c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:31.946741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b3a98-dbdd-4a17-9973-ecdc13842631 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:31.988503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a723e-869b-4072-b074-87c52c6db8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:31.989126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a723e-869b-4072-b074-87c52c6db8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:31.989404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a723e-869b-4072-b074-87c52c6db8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:31.989706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a723e-869b-4072-b074-87c52c6db8f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:31.991423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a723e-869b-4072-b074-87c52c6db8f4 None admin] RBAC: Adding query filter params (interface=public, service_id=3029c7008e7f44e7ab512ebe24c8f051, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:31.991740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a723e-869b-4072-b074-87c52c6db8f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:31.993120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a723e-869b-4072-b074-87c52c6db8f4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:31.996420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a612e-b1d8-486f-a551-b4d371950d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:31.997070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a612e-b1d8-486f-a551-b4d371950d02 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:31.997353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a612e-b1d8-486f-a551-b4d371950d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:31.997816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a612e-b1d8-486f-a551-b4d371950d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:31.998781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 490/983] 10.222.0.100 () {56 vars in 1256 bytes} [Thu Sep 1 17:24:31 2022] GET /identity/v3/endpoints?service_id=3029c7008e7f44e7ab512ebe24c8f051&interface=public®ion_id=RegionOne => generated 195 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:32.006220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a723e-869b-4072-b074-87c52c6db8f4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:32.015550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a612e-b1d8-486f-a551-b4d371950d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:32.017020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a612e-b1d8-486f-a551-b4d371950d02 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e43a612e-b1d8-486f-a551-b4d371950d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:32.037046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:32.038124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:32.039033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:32.039033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:32.039033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:32.039033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 494/984] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:32.044232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a612e-b1d8-486f-a551-b4d371950d02 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:32.066134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:32.126687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a723e-869b-4072-b074-87c52c6db8f4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:32.156471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:32.159697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a612e-b1d8-486f-a551-b4d371950d02 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:32.168654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfaa26-ecd8-4acd-b1bf-c9ca8f623dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:32.169266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfaa26-ecd8-4acd-b1bf-c9ca8f623dfe None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:32.169550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfaa26-ecd8-4acd-b1bf-c9ca8f623dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:32.169851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfaa26-ecd8-4acd-b1bf-c9ca8f623dfe None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:32.170581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfaa26-ecd8-4acd-b1bf-c9ca8f623dfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:32.172088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfaa26-ecd8-4acd-b1bf-c9ca8f623dfe None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:32.186721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 491/985] 10.222.0.100 () {56 vars in 1092 bytes} [Thu Sep 1 17:24:32 2022] GET /identity/v3/services => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:24:32.210888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e834b4c-3c99-482a-adf3-3b0235ef41e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:32.211626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e834b4c-3c99-482a-adf3-3b0235ef41e0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:32.211980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e834b4c-3c99-482a-adf3-3b0235ef41e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:32.212326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e834b4c-3c99-482a-adf3-3b0235ef41e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:32.213214 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e834b4c-3c99-482a-adf3-3b0235ef41e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:32.213464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e834b4c-3c99-482a-adf3-3b0235ef41e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:32.215004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e834b4c-3c99-482a-adf3-3b0235ef41e0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:32.236162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 495/986] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:32.237675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfaa26-ecd8-4acd-b1bf-c9ca8f623dfe None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:32.360469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfaa26-ecd8-4acd-b1bf-c9ca8f623dfe None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:32.373284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:32.403309 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc6dbb-99c2-45c3-bce9-a3575ba47bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:32.404438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc6dbb-99c2-45c3-bce9-a3575ba47bbb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:32.404929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc6dbb-99c2-45c3-bce9-a3575ba47bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:32.405436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc6dbb-99c2-45c3-bce9-a3575ba47bbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:32.416588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc6dbb-99c2-45c3-bce9-a3575ba47bbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:32.418684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc6dbb-99c2-45c3-bce9-a3575ba47bbb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b0fc6dbb-99c2-45c3-bce9-a3575ba47bbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:32.432379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:32.433430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:32.434365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:32.434365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:32.434365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:32.434365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:32.434365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:32.434365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:32.434365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:32.434365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:32.434787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 492/987] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:32.440240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e834b4c-3c99-482a-adf3-3b0235ef41e0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:32.558256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e834b4c-3c99-482a-adf3-3b0235ef41e0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:32.572559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1028b5b-62e8-448f-8c44-f8cb1777929b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:32.572865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1028b5b-62e8-448f-8c44-f8cb1777929b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:32.573241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1028b5b-62e8-448f-8c44-f8cb1777929b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:32.574720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 493/988] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:32.580669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa77337-d010-449d-9e3d-0f9fb0e8d693 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:32.580948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa77337-d010-449d-9e3d-0f9fb0e8d693 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:32.581238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa77337-d010-449d-9e3d-0f9fb0e8d693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:32.603319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ad94-ffe5-4dcb-8cc9-440ab6ecd6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:32.604187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ad94-ffe5-4dcb-8cc9-440ab6ecd6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:32.604475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ad94-ffe5-4dcb-8cc9-440ab6ecd6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:32.604573 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8aa77337-d010-449d-9e3d-0f9fb0e8d693 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:32.604754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ad94-ffe5-4dcb-8cc9-440ab6ecd6b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:32.605472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73ad94-ffe5-4dcb-8cc9-440ab6ecd6b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:32.605813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73ad94-ffe5-4dcb-8cc9-440ab6ecd6b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:32.605938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8aa77337-d010-449d-9e3d-0f9fb0e8d693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:32.607274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73ad94-ffe5-4dcb-8cc9-440ab6ecd6b5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:32.623395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 496/989] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:32.629472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ad94-ffe5-4dcb-8cc9-440ab6ecd6b5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:32.665494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:32.708263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 494/990] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:32 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:32.773042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ad94-ffe5-4dcb-8cc9-440ab6ecd6b5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:32.812449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a39e23-6cad-409b-80cf-eac3c3383c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:32.813057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a39e23-6cad-409b-80cf-eac3c3383c92 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:32.813341 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a39e23-6cad-409b-80cf-eac3c3383c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:32.813633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a39e23-6cad-409b-80cf-eac3c3383c92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:32.814366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a39e23-6cad-409b-80cf-eac3c3383c92 None admin] RBAC: Adding query filter params (role.id=42a25be79662491da6d022c8d6fb860f, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=71b9335f3f744a31a84ab278d9b45e36) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:32.814677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a39e23-6cad-409b-80cf-eac3c3383c92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:32.816108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a39e23-6cad-409b-80cf-eac3c3383c92 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:32.829891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 497/991] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:32 2022] GET /identity/v3/role_assignments?user.id=71b9335f3f744a31a84ab278d9b45e36&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=42a25be79662491da6d022c8d6fb860f => generated 610 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:33.000127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa77337-d010-449d-9e3d-0f9fb0e8d693 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:33.122437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:33.127984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa77337-d010-449d-9e3d-0f9fb0e8d693 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:33.204708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5cd48-3563-4199-8c44-00ebc351d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:33.205834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5cd48-3563-4199-8c44-00ebc351d123 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:33.206289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5cd48-3563-4199-8c44-00ebc351d123 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:33.206732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5cd48-3563-4199-8c44-00ebc351d123 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:33.207767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5cd48-3563-4199-8c44-00ebc351d123 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:33.211286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5cd48-3563-4199-8c44-00ebc351d123 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f2a5cd48-3563-4199-8c44-00ebc351d123 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:33.229402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:33.230652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:33.231776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:33.233247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 495/992] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:33.241837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a39e23-6cad-409b-80cf-eac3c3383c92 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:33.361644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:33.389450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a39e23-6cad-409b-80cf-eac3c3383c92 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:33.430560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b6102-d122-4fa2-a1dd-3fdd76ce7334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:33.431442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b6102-d122-4fa2-a1dd-3fdd76ce7334 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:33.431874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b6102-d122-4fa2-a1dd-3fdd76ce7334 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:33.432325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b6102-d122-4fa2-a1dd-3fdd76ce7334 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:33.433128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b6102-d122-4fa2-a1dd-3fdd76ce7334 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:33.433674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b6102-d122-4fa2-a1dd-3fdd76ce7334 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:33.435347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b6102-d122-4fa2-a1dd-3fdd76ce7334 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:33.445772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 498/993] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:33 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:33.449527 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5cd48-3563-4199-8c44-00ebc351d123 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:33.570275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5cd48-3563-4199-8c44-00ebc351d123 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:33.617369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be0b85-634f-4ee0-95b7-61aac147c50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:33.618441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0b85-634f-4ee0-95b7-61aac147c50f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:33.619427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0b85-634f-4ee0-95b7-61aac147c50f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:33.620130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0b85-634f-4ee0-95b7-61aac147c50f None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:33.620945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be0b85-634f-4ee0-95b7-61aac147c50f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:33.621392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be0b85-634f-4ee0-95b7-61aac147c50f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:33.623069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be0b85-634f-4ee0-95b7-61aac147c50f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:33.629241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 496/994] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:33 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:33.637353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b6102-d122-4fa2-a1dd-3fdd76ce7334 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:33.759588 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b6102-d122-4fa2-a1dd-3fdd76ce7334 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:33.803855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cbe33-13d2-4b5b-8254-5eb92734c1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:33.804473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cbe33-13d2-4b5b-8254-5eb92734c1fd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:33.804771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cbe33-13d2-4b5b-8254-5eb92734c1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:33.805030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cbe33-13d2-4b5b-8254-5eb92734c1fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:33.805749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cbe33-13d2-4b5b-8254-5eb92734c1fd None admin] RBAC: Adding query filter params (interface=admin, service_id=84bd851758614e7f8678ae6eb88f65dc, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:33.806055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cbe33-13d2-4b5b-8254-5eb92734c1fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:33.807409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cbe33-13d2-4b5b-8254-5eb92734c1fd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:33.813441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 499/995] 10.222.0.100 () {56 vars in 1254 bytes} [Thu Sep 1 17:24:33 2022] GET /identity/v3/endpoints?service_id=84bd851758614e7f8678ae6eb88f65dc&interface=admin®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:33.819251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be0b85-634f-4ee0-95b7-61aac147c50f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:33.854672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:33.954314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be0b85-634f-4ee0-95b7-61aac147c50f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:33.994003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd3cb-ab46-4759-b14e-3bcf52effb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:33.994674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd3cb-ab46-4759-b14e-3bcf52effb6f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:33.995131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd3cb-ab46-4759-b14e-3bcf52effb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:33.995584 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd3cb-ab46-4759-b14e-3bcf52effb6f None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:33.996374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd3cb-ab46-4759-b14e-3bcf52effb6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:33.997747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd3cb-ab46-4759-b14e-3bcf52effb6f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:34.005472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 497/996] 10.222.0.100 () {56 vars in 1092 bytes} [Thu Sep 1 17:24:33 2022] GET /identity/v3/services => generated 1735 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:24:34.198669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9042ca-6545-48d9-b49f-6971257ac5cf None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:34.198952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9042ca-6545-48d9-b49f-6971257ac5cf None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:34.199233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9042ca-6545-48d9-b49f-6971257ac5cf None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:34.200619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 500/997] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:34.206611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f3fde-f6d2-4c67-b2a4-d8f1bf00f9c5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:34.206823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f3fde-f6d2-4c67-b2a4-d8f1bf00f9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:34.207107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f3fde-f6d2-4c67-b2a4-d8f1bf00f9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:34.227794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2d5f3fde-f6d2-4c67-b2a4-d8f1bf00f9c5 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:34.228993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d5f3fde-f6d2-4c67-b2a4-d8f1bf00f9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:34.303325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 498/998] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:34 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:34.574924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9042ca-6545-48d9-b49f-6971257ac5cf None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:34.591711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:34.699057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9042ca-6545-48d9-b49f-6971257ac5cf None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:34.745727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2753-765a-4da3-8663-e6d50596e5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:34.746361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2753-765a-4da3-8663-e6d50596e5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:34.746643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2753-765a-4da3-8663-e6d50596e5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:34.746917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2753-765a-4da3-8663-e6d50596e5f1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:34.748173 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2753-765a-4da3-8663-e6d50596e5f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:34.748874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2753-765a-4da3-8663-e6d50596e5f1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-fcfb2753-765a-4da3-8663-e6d50596e5f1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:34.759144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:34.760261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:24:34.761103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:34.765259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 501/999] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:34.766970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f3fde-f6d2-4c67-b2a4-d8f1bf00f9c5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:34.801555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:34.870568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d6ac5-cb49-4903-af30-cb33e757f8af None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:34.870823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d6ac5-cb49-4903-af30-cb33e757f8af None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:34.870990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d6ac5-cb49-4903-af30-cb33e757f8af None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:34.872333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 502/1000] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:34.877962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490478f3-363d-4df6-9973-027dbc0b31da None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:34.878246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490478f3-363d-4df6-9973-027dbc0b31da None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:34.878521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490478f3-363d-4df6-9973-027dbc0b31da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:34.897936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-490478f3-363d-4df6-9973-027dbc0b31da None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:34.899123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-490478f3-363d-4df6-9973-027dbc0b31da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:34.901121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f3fde-f6d2-4c67-b2a4-d8f1bf00f9c5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:34.945418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ea738-01dd-4d31-9643-6018b5cb265f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:34.946113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea738-01dd-4d31-9643-6018b5cb265f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:34.946398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea738-01dd-4d31-9643-6018b5cb265f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:34.946668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea738-01dd-4d31-9643-6018b5cb265f None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:34.947632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ea738-01dd-4d31-9643-6018b5cb265f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:34.947632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ea738-01dd-4d31-9643-6018b5cb265f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:34.949003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ea738-01dd-4d31-9643-6018b5cb265f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:34.959971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 499/1001] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:34 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:34.961672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ea738-01dd-4d31-9643-6018b5cb265f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:34.977974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 503/1002] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:34 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:35.066567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:35.085437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ea738-01dd-4d31-9643-6018b5cb265f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:35.133744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e6480-e3b7-4b84-b7bb-a1533d01dbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:35.134304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e6480-e3b7-4b84-b7bb-a1533d01dbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:35.134579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e6480-e3b7-4b84-b7bb-a1533d01dbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:35.134850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e6480-e3b7-4b84-b7bb-a1533d01dbb4 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:35.135516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e6480-e3b7-4b84-b7bb-a1533d01dbb4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:35.135820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e6480-e3b7-4b84-b7bb-a1533d01dbb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:35.137227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e6480-e3b7-4b84-b7bb-a1533d01dbb4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:35.143080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 500/1003] 10.222.0.100 () {56 vars in 1117 bytes} [Thu Sep 1 17:24:34 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:35.149420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490478f3-363d-4df6-9973-027dbc0b31da None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:35.159191 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:35.254011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e6480-e3b7-4b84-b7bb-a1533d01dbb4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:35.267294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490478f3-363d-4df6-9973-027dbc0b31da None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:35.311095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813e23a-e2f0-44ae-a20b-23d08c29467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:35.311833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813e23a-e2f0-44ae-a20b-23d08c29467f None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:35.312108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813e23a-e2f0-44ae-a20b-23d08c29467f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:35.312383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813e23a-e2f0-44ae-a20b-23d08c29467f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:35.313071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813e23a-e2f0-44ae-a20b-23d08c29467f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:35.314499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813e23a-e2f0-44ae-a20b-23d08c29467f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:35.350311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 504/1004] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:24:35 2022] POST /identity/v3/endpoints => generated 336 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:35.405062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e6480-e3b7-4b84-b7bb-a1533d01dbb4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:35.461315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15578fbc-e49a-4466-ae3b-f84e6932b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:35.462005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15578fbc-e49a-4466-ae3b-f84e6932b3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:35.462283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15578fbc-e49a-4466-ae3b-f84e6932b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:35.462553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15578fbc-e49a-4466-ae3b-f84e6932b3d9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:35.463147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15578fbc-e49a-4466-ae3b-f84e6932b3d9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:35.464582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15578fbc-e49a-4466-ae3b-f84e6932b3d9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-15578fbc-e49a-4466-ae3b-f84e6932b3d9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:35.475319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:35.476525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:35.482824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 501/1005] 10.222.0.100 () {56 vars in 1110 bytes} [Thu Sep 1 17:24:35 2022] GET /identity/v3/services/image => generated 86 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:35.483647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813e23a-e2f0-44ae-a20b-23d08c29467f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:35.567568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:35.617479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813e23a-e2f0-44ae-a20b-23d08c29467f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:35.664120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de250610-e359-4828-98d1-2d80f6b0add0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:35.664835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de250610-e359-4828-98d1-2d80f6b0add0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:35.665033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de250610-e359-4828-98d1-2d80f6b0add0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:35.665418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de250610-e359-4828-98d1-2d80f6b0add0 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:35.665963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de250610-e359-4828-98d1-2d80f6b0add0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:35.666262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de250610-e359-4828-98d1-2d80f6b0add0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:35.667648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de250610-e359-4828-98d1-2d80f6b0add0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:35.673644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 505/1006] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:35 2022] GET /identity/v3/services?name=image => generated 123 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:35.679299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15578fbc-e49a-4466-ae3b-f84e6932b3d9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:35.693617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:35.800501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15578fbc-e49a-4466-ae3b-f84e6932b3d9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:35.837899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea3f96-789d-4ec0-a656-872e35e64613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:35.838512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3f96-789d-4ec0-a656-872e35e64613 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:35.838786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3f96-789d-4ec0-a656-872e35e64613 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:35.839050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3f96-789d-4ec0-a656-872e35e64613 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:35.839715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea3f96-789d-4ec0-a656-872e35e64613 None admin] RBAC: Adding query filter params (type=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:35.840009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea3f96-789d-4ec0-a656-872e35e64613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:35.841377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea3f96-789d-4ec0-a656-872e35e64613 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:35.847139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 502/1007] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:35 2022] GET /identity/v3/services?type=image => generated 123 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:36.117814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef62dcb-f826-4519-a0f1-ec5c1cb254f4 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:36.118165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef62dcb-f826-4519-a0f1-ec5c1cb254f4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:36.118562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef62dcb-f826-4519-a0f1-ec5c1cb254f4 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:36.120436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 506/1008] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:36 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:36.126337 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ceb6ac-bc12-4d2f-acf5-1b0234e4f244 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:36.126620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ceb6ac-bc12-4d2f-acf5-1b0234e4f244 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:36.126980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ceb6ac-bc12-4d2f-acf5-1b0234e4f244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:36.146919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-23ceb6ac-bc12-4d2f-acf5-1b0234e4f244 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:36.147910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-23ceb6ac-bc12-4d2f-acf5-1b0234e4f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:36.226553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 503/1009] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:36 2022] POST /identity/v3/auth/tokens => generated 1729 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:36.501334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef62dcb-f826-4519-a0f1-ec5c1cb254f4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:36.641202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef62dcb-f826-4519-a0f1-ec5c1cb254f4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:36.692913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:36.709641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2f113-63ac-4b4c-9f32-bf134ebe3c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:36.710933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2f113-63ac-4b4c-9f32-bf134ebe3c87 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:36.711391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2f113-63ac-4b4c-9f32-bf134ebe3c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:36.711811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2f113-63ac-4b4c-9f32-bf134ebe3c87 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:36.712798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2f113-63ac-4b4c-9f32-bf134ebe3c87 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:36.715254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2f113-63ac-4b4c-9f32-bf134ebe3c87 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-49e2f113-63ac-4b4c-9f32-bf134ebe3c87 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:36.732202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:36.733419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:36.738747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 507/1010] 10.222.0.100 () {56 vars in 1131 bytes} [Thu Sep 1 17:24:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:36.740538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ceb6ac-bc12-4d2f-acf5-1b0234e4f244 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:36.744958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:36.880400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ceb6ac-bc12-4d2f-acf5-1b0234e4f244 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:36.923202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60312a82-3458-42fd-8fc1-d07ab31f6de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:36.923825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60312a82-3458-42fd-8fc1-d07ab31f6de6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:36.924101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60312a82-3458-42fd-8fc1-d07ab31f6de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:36.924366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60312a82-3458-42fd-8fc1-d07ab31f6de6 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:36.925002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60312a82-3458-42fd-8fc1-d07ab31f6de6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:36.925300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60312a82-3458-42fd-8fc1-d07ab31f6de6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:36.926748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60312a82-3458-42fd-8fc1-d07ab31f6de6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:36.937626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 504/1011] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:36 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:36.941246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2f113-63ac-4b4c-9f32-bf134ebe3c87 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:37.081840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2f113-63ac-4b4c-9f32-bf134ebe3c87 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:37.139881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581bcd6-bc22-4ebc-84ec-e5b7a10e5b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:37.140533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581bcd6-bc22-4ebc-84ec-e5b7a10e5b35 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:37.140791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581bcd6-bc22-4ebc-84ec-e5b7a10e5b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:37.141060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581bcd6-bc22-4ebc-84ec-e5b7a10e5b35 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:37.141732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581bcd6-bc22-4ebc-84ec-e5b7a10e5b35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:37.142033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581bcd6-bc22-4ebc-84ec-e5b7a10e5b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:37.143435 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581bcd6-bc22-4ebc-84ec-e5b7a10e5b35 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:37.150727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 508/1012] 10.222.0.100 () {56 vars in 1127 bytes} [Thu Sep 1 17:24:36 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:37.156732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60312a82-3458-42fd-8fc1-d07ab31f6de6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:37.244059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:37.286033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60312a82-3458-42fd-8fc1-d07ab31f6de6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:37.325257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ba07d-4bb1-4c13-b83b-d6bd8d9a9114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:37.325917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba07d-4bb1-4c13-b83b-d6bd8d9a9114 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:37.326190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba07d-4bb1-4c13-b83b-d6bd8d9a9114 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:37.326457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba07d-4bb1-4c13-b83b-d6bd8d9a9114 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:37.327139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ba07d-4bb1-4c13-b83b-d6bd8d9a9114 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:37.328522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ba07d-4bb1-4c13-b83b-d6bd8d9a9114 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:37.348488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 505/1013] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:24:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:24:37.480258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf2bea-3e8c-4b4a-a18b-a100e1e0628a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:37.480461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf2bea-3e8c-4b4a-a18b-a100e1e0628a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:37.480732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf2bea-3e8c-4b4a-a18b-a100e1e0628a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:37.482340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 509/1014] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:37.488057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437c506-5d68-46f6-b941-e3d2e4f7d761 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:37.488474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437c506-5d68-46f6-b941-e3d2e4f7d761 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:37.488602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437c506-5d68-46f6-b941-e3d2e4f7d761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:37.508236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c437c506-5d68-46f6-b941-e3d2e4f7d761 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:37.509447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c437c506-5d68-46f6-b941-e3d2e4f7d761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:37.617773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 506/1015] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:37 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:37.852696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712855d-a3b7-44c1-a7fa-6d47c78a4520 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:37.852696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712855d-a3b7-44c1-a7fa-6d47c78a4520 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:37.852696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712855d-a3b7-44c1-a7fa-6d47c78a4520 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:37.853388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 510/1016] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:37.858555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892110be-c825-4137-95cb-7923ba8ab2bb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:37.858925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892110be-c825-4137-95cb-7923ba8ab2bb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:37.859135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892110be-c825-4137-95cb-7923ba8ab2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:37.879452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-892110be-c825-4137-95cb-7923ba8ab2bb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:37.881221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-892110be-c825-4137-95cb-7923ba8ab2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:37.899674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712855d-a3b7-44c1-a7fa-6d47c78a4520 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:37.962243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:37.976363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:37.992136 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 507/1017] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:37 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:38.064990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712855d-a3b7-44c1-a7fa-6d47c78a4520 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:38.133711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385739e-2daa-4353-ae07-8716076c66b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:38.134693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385739e-2daa-4353-ae07-8716076c66b1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:38.134981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385739e-2daa-4353-ae07-8716076c66b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:38.135360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385739e-2daa-4353-ae07-8716076c66b1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:38.154896 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385739e-2daa-4353-ae07-8716076c66b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e385739e-2daa-4353-ae07-8716076c66b1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:38.164939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:38.166366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:38.170893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 511/1018] 10.222.0.100 () {56 vars in 1125 bytes} [Thu Sep 1 17:24:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:38.178768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892110be-c825-4137-95cb-7923ba8ab2bb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:38.263310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385739e-2daa-4353-ae07-8716076c66b1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:38.328224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892110be-c825-4137-95cb-7923ba8ab2bb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:38.386494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55715672-44c5-4e4c-a2b0-5be9c109ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:38.387449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55715672-44c5-4e4c-a2b0-5be9c109ce4f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:38.387961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55715672-44c5-4e4c-a2b0-5be9c109ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:38.388437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55715672-44c5-4e4c-a2b0-5be9c109ce4f None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:38.389736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55715672-44c5-4e4c-a2b0-5be9c109ce4f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:38.390281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55715672-44c5-4e4c-a2b0-5be9c109ce4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:38.392937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55715672-44c5-4e4c-a2b0-5be9c109ce4f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:38.401603 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385739e-2daa-4353-ae07-8716076c66b1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:38.413640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 508/1019] 10.222.0.100 () {56 vars in 1120 bytes} [Thu Sep 1 17:24:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:38.419397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55715672-44c5-4e4c-a2b0-5be9c109ce4f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:38.427445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:38.463490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32f0b5-2be1-4452-8e0a-26037ce7c99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:38.464114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32f0b5-2be1-4452-8e0a-26037ce7c99f None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:38.464393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32f0b5-2be1-4452-8e0a-26037ce7c99f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:38.464667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32f0b5-2be1-4452-8e0a-26037ce7c99f None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:38.465348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32f0b5-2be1-4452-8e0a-26037ce7c99f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:38.466776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32f0b5-2be1-4452-8e0a-26037ce7c99f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:38.485127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 512/1020] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:24:38 2022] POST /identity/v3/services => generated 243 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:24:38.486556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:38.594788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55715672-44c5-4e4c-a2b0-5be9c109ce4f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:38.671678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdc537-0840-41fa-8eb1-020a24c67dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:38.672327 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdc537-0840-41fa-8eb1-020a24c67dda None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:38.672606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdc537-0840-41fa-8eb1-020a24c67dda None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:38.672877 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdc537-0840-41fa-8eb1-020a24c67dda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:38.685907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdc537-0840-41fa-8eb1-020a24c67dda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:38.688194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdc537-0840-41fa-8eb1-020a24c67dda None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e2bdc537-0840-41fa-8eb1-020a24c67dda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:38.705087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:38.706780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:38.708242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:38.708242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:38.708242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:38.708242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 509/1021] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:38.719303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32f0b5-2be1-4452-8e0a-26037ce7c99f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:38.831238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32f0b5-2be1-4452-8e0a-26037ce7c99f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:38.867607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a12c75-ce49-4d3f-bdc5-28994cf30e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:38.868609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a12c75-ce49-4d3f-bdc5-28994cf30e9e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:38.868609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a12c75-ce49-4d3f-bdc5-28994cf30e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:38.868754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a12c75-ce49-4d3f-bdc5-28994cf30e9e None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:38.869635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a12c75-ce49-4d3f-bdc5-28994cf30e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:38.869762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a12c75-ce49-4d3f-bdc5-28994cf30e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:38.871034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a12c75-ce49-4d3f-bdc5-28994cf30e9e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:38.889662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 513/1022] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:38.897910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdc537-0840-41fa-8eb1-020a24c67dda None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:39.049944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:39.060752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdc537-0840-41fa-8eb1-020a24c67dda None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:39.097349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902210a9-4da7-473a-baf5-91fb4dc2f2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:39.098001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902210a9-4da7-473a-baf5-91fb4dc2f2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:39.098275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902210a9-4da7-473a-baf5-91fb4dc2f2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:39.098541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902210a9-4da7-473a-baf5-91fb4dc2f2b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:39.108673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902210a9-4da7-473a-baf5-91fb4dc2f2b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:39.110273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902210a9-4da7-473a-baf5-91fb4dc2f2b5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-902210a9-4da7-473a-baf5-91fb4dc2f2b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:39.122966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:39.124153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:39.125117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 510/1023] 10.222.0.100 () {56 vars in 1098 bytes} [Thu Sep 1 17:24:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:39.129831 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a12c75-ce49-4d3f-bdc5-28994cf30e9e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:39.167353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:39.239455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a12c75-ce49-4d3f-bdc5-28994cf30e9e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:39.274899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d172a-556b-4f9a-a56d-8d3e1cd92884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:39.275507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d172a-556b-4f9a-a56d-8d3e1cd92884 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:39.275779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d172a-556b-4f9a-a56d-8d3e1cd92884 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:39.276045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d172a-556b-4f9a-a56d-8d3e1cd92884 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:39.276780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d172a-556b-4f9a-a56d-8d3e1cd92884 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:39.277081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d172a-556b-4f9a-a56d-8d3e1cd92884 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:39.278820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d172a-556b-4f9a-a56d-8d3e1cd92884 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:39.313429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 514/1024] 10.222.0.100 () {56 vars in 1138 bytes} [Thu Sep 1 17:24:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:39.314876 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902210a9-4da7-473a-baf5-91fb4dc2f2b5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:39.407608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64e696-8463-4b52-a932-d836c760b458 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:39.408014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64e696-8463-4b52-a932-d836c760b458 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:39.408168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64e696-8463-4b52-a932-d836c760b458 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:39.409529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 515/1025] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:39.415517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dde9b-ad9d-4d8f-b6e0-583ca2b23408 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:39.415795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dde9b-ad9d-4d8f-b6e0-583ca2b23408 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:39.416066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dde9b-ad9d-4d8f-b6e0-583ca2b23408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:39.436217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b92dde9b-ad9d-4d8f-b6e0-583ca2b23408 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:39.437409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b92dde9b-ad9d-4d8f-b6e0-583ca2b23408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:39.439675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902210a9-4da7-473a-baf5-91fb4dc2f2b5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:39.477436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0648b-ea26-4d36-80de-5eefc2e710ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:39.478398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0648b-ea26-4d36-80de-5eefc2e710ab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:39.478675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0648b-ea26-4d36-80de-5eefc2e710ab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:39.478947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0648b-ea26-4d36-80de-5eefc2e710ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:39.507426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:39.522116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 516/1026] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:39 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:39.564978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0648b-ea26-4d36-80de-5eefc2e710ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:39.566383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0648b-ea26-4d36-80de-5eefc2e710ab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:39.687496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 511/1027] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:39 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:24:39.689254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dde9b-ad9d-4d8f-b6e0-583ca2b23408 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:39.717975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:39.799314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0648b-ea26-4d36-80de-5eefc2e710ab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:39.808148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dde9b-ad9d-4d8f-b6e0-583ca2b23408 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:39.857665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a226b3-66b5-4b13-a5dd-b738d7cdaa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:39.858233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a226b3-66b5-4b13-a5dd-b738d7cdaa0f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:39.858517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a226b3-66b5-4b13-a5dd-b738d7cdaa0f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:39.858871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a226b3-66b5-4b13-a5dd-b738d7cdaa0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:39.867805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:39.872777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a226b3-66b5-4b13-a5dd-b738d7cdaa0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:39.874345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a226b3-66b5-4b13-a5dd-b738d7cdaa0f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e1a226b3-66b5-4b13-a5dd-b738d7cdaa0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:39.890712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:39.900099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:39.901956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:39.901956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:39.901956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:39.901956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 517/1028] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:39.904805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a226b3-66b5-4b13-a5dd-b738d7cdaa0f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:39.933746 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0648b-ea26-4d36-80de-5eefc2e710ab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:39.978543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a587f9-0828-4db6-a510-42309c0c9d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:39.979186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a587f9-0828-4db6-a510-42309c0c9d2e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:39.979464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a587f9-0828-4db6-a510-42309c0c9d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:39.979737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a587f9-0828-4db6-a510-42309c0c9d2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:39.980854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a587f9-0828-4db6-a510-42309c0c9d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:39.982317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a587f9-0828-4db6-a510-42309c0c9d2e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:39.983914 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 512/1029] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:39 2022] GET /identity/v3/domains/default => generated 200 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:39.989646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a587f9-0828-4db6-a510-42309c0c9d2e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:39.993526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:40.026434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a226b3-66b5-4b13-a5dd-b738d7cdaa0f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:40.069995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f7cd1-d7d3-4579-93c3-f092fcec2c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:40.070733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f7cd1-d7d3-4579-93c3-f092fcec2c59 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:40.071011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f7cd1-d7d3-4579-93c3-f092fcec2c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:40.071282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f7cd1-d7d3-4579-93c3-f092fcec2c59 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:40.071972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f7cd1-d7d3-4579-93c3-f092fcec2c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:40.072303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f7cd1-d7d3-4579-93c3-f092fcec2c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:40.073802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f7cd1-d7d3-4579-93c3-f092fcec2c59 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:40.087123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 518/1030] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:40.092908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f7cd1-d7d3-4579-93c3-f092fcec2c59 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:40.116686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a587f9-0828-4db6-a510-42309c0c9d2e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:40.143784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:40.157946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362912-8a04-472b-90ae-bbe0a4b87eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:40.158627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362912-8a04-472b-90ae-bbe0a4b87eac None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:40.158913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362912-8a04-472b-90ae-bbe0a4b87eac None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:40.159186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362912-8a04-472b-90ae-bbe0a4b87eac None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:40.159846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362912-8a04-472b-90ae-bbe0a4b87eac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:40.161291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362912-8a04-472b-90ae-bbe0a4b87eac None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:40.192250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 513/1031] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:24:39 2022] POST /identity/v3/projects => generated 312 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:24:40.207844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f7cd1-d7d3-4579-93c3-f092fcec2c59 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:40.250021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7e9b6-3edd-4517-8078-cd0363125922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:40.250679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7e9b6-3edd-4517-8078-cd0363125922 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:40.250957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7e9b6-3edd-4517-8078-cd0363125922 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:40.251226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7e9b6-3edd-4517-8078-cd0363125922 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:40.253543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:40.262033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7e9b6-3edd-4517-8078-cd0363125922 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:40.264046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7e9b6-3edd-4517-8078-cd0363125922 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e6f7e9b6-3edd-4517-8078-cd0363125922 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:40.277519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:40.278843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:40.284978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:40.284978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:40.284978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:40.284978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:40.284978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:40.284978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:40.284978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:40.284978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:40.284978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 519/1032] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:40.287305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362912-8a04-472b-90ae-bbe0a4b87eac None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:40.409252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362912-8a04-472b-90ae-bbe0a4b87eac None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:40.463929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12700c11-9090-4967-aa70-cb00f3284a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:40.467249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12700c11-9090-4967-aa70-cb00f3284a19 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:40.467739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12700c11-9090-4967-aa70-cb00f3284a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:40.468052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12700c11-9090-4967-aa70-cb00f3284a19 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:40.469352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12700c11-9090-4967-aa70-cb00f3284a19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:40.470476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12700c11-9090-4967-aa70-cb00f3284a19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:40.473237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12700c11-9090-4967-aa70-cb00f3284a19 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:40.496448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 514/1033] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:40.502821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7e9b6-3edd-4517-8078-cd0363125922 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:40.517826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bfd04-cdfd-4f78-974a-f1b651f06e04 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:40.518464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bfd04-cdfd-4f78-974a-f1b651f06e04 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:40.519088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bfd04-cdfd-4f78-974a-f1b651f06e04 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:40.522452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 515/1034] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:40 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:40.527907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ff3f1-8511-4c30-aa79-bcb93386eff8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:40.528584 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ff3f1-8511-4c30-aa79-bcb93386eff8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:40.528956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ff3f1-8511-4c30-aa79-bcb93386eff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:40.549063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-906ff3f1-8511-4c30-aa79-bcb93386eff8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:40.551224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-906ff3f1-8511-4c30-aa79-bcb93386eff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:40.590062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:40.636047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7e9b6-3edd-4517-8078-cd0363125922 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:40.640199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 516/1035] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:40 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:40.685228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79879b83-f8f4-4446-95cb-1add7e1c028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:40.686230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79879b83-f8f4-4446-95cb-1add7e1c028f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:40.686746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79879b83-f8f4-4446-95cb-1add7e1c028f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:40.687234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79879b83-f8f4-4446-95cb-1add7e1c028f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:40.688391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79879b83-f8f4-4446-95cb-1add7e1c028f None admin] RBAC: Adding query filter params (role.id=42a25be79662491da6d022c8d6fb860f, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:40.688703 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79879b83-f8f4-4446-95cb-1add7e1c028f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:40.690771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79879b83-f8f4-4446-95cb-1add7e1c028f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:40.704789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 520/1036] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:40 2022] GET /identity/v3/role_assignments?user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=42a25be79662491da6d022c8d6fb860f => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:40.919441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ff3f1-8511-4c30-aa79-bcb93386eff8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:41.035915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ff3f1-8511-4c30-aa79-bcb93386eff8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:41.086599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26192cf3-db90-4020-8052-f9e45373f354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:41.087666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26192cf3-db90-4020-8052-f9e45373f354 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:41.088192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26192cf3-db90-4020-8052-f9e45373f354 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:41.088619 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26192cf3-db90-4020-8052-f9e45373f354 None admin] PATH_INFO: `/v3/services/image` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:41.089694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26192cf3-db90-4020-8052-f9e45373f354 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:41.092313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26192cf3-db90-4020-8052-f9e45373f354 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-26192cf3-db90-4020-8052-f9e45373f354 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:41.109768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:41.110944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:24:41.111878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:41.113426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 517/1037] 10.222.0.100 () {56 vars in 1110 bytes} [Thu Sep 1 17:24:40 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:41.121716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79879b83-f8f4-4446-95cb-1add7e1c028f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:41.226519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:41.280283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79879b83-f8f4-4446-95cb-1add7e1c028f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:41.319783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb008412-426c-453c-8859-ed23185a8d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:41.320307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb008412-426c-453c-8859-ed23185a8d1a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:41.320592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb008412-426c-453c-8859-ed23185a8d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:41.320866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb008412-426c-453c-8859-ed23185a8d1a None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:41.321513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb008412-426c-453c-8859-ed23185a8d1a None admin] RBAC: Adding query filter params (name=image) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:41.321858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb008412-426c-453c-8859-ed23185a8d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:41.323273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb008412-426c-453c-8859-ed23185a8d1a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:41.329432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 521/1038] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:41 2022] GET /identity/v3/services?name=image => generated 123 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:41.336079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26192cf3-db90-4020-8052-f9e45373f354 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:41.369864 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:41.460818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26192cf3-db90-4020-8052-f9e45373f354 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:41.507293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acd58e-8619-40e3-97cf-f458b3c3fb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:41.507918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acd58e-8619-40e3-97cf-f458b3c3fb1f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:41.508292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acd58e-8619-40e3-97cf-f458b3c3fb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:41.508588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acd58e-8619-40e3-97cf-f458b3c3fb1f None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:41.509253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acd58e-8619-40e3-97cf-f458b3c3fb1f None admin] RBAC: Adding query filter params (type=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:41.509561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acd58e-8619-40e3-97cf-f458b3c3fb1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:41.510985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acd58e-8619-40e3-97cf-f458b3c3fb1f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:41.523874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 518/1039] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:41 2022] GET /identity/v3/services?type=image => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:41.525510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb008412-426c-453c-8859-ed23185a8d1a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:41.644998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb008412-426c-453c-8859-ed23185a8d1a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:41.678444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:41.697704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf2fcc-4f5b-4d6c-b674-d9e4c59365cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:41.698595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2fcc-4f5b-4d6c-b674-d9e4c59365cd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:41.699016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2fcc-4f5b-4d6c-b674-d9e4c59365cd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:41.699321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2fcc-4f5b-4d6c-b674-d9e4c59365cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:41.700147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf2fcc-4f5b-4d6c-b674-d9e4c59365cd None admin] RBAC: Adding query filter params (interface=public, service_id=be5077ecef144bf7a49f0b250117c35f, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:41.700658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf2fcc-4f5b-4d6c-b674-d9e4c59365cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:41.703157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf2fcc-4f5b-4d6c-b674-d9e4c59365cd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:41.716742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 522/1040] 10.222.0.100 () {56 vars in 1256 bytes} [Thu Sep 1 17:24:41 2022] GET /identity/v3/endpoints?service_id=be5077ecef144bf7a49f0b250117c35f&interface=public®ion_id=RegionOne => generated 195 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:41.718502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acd58e-8619-40e3-97cf-f458b3c3fb1f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:41.872314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:41.885214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acd58e-8619-40e3-97cf-f458b3c3fb1f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:41.930056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd3969-7562-499e-9ff5-00eabe86e2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:41.930591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd3969-7562-499e-9ff5-00eabe86e2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:41.930874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd3969-7562-499e-9ff5-00eabe86e2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:41.931142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd3969-7562-499e-9ff5-00eabe86e2e0 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:41.931752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd3969-7562-499e-9ff5-00eabe86e2e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:41.933135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd3969-7562-499e-9ff5-00eabe86e2e0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:41.945679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 519/1041] 10.222.0.100 () {56 vars in 1092 bytes} [Thu Sep 1 17:24:41 2022] GET /identity/v3/services => generated 1966 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:24:42.299026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b221c3-9c1c-4209-8e0c-871f43ee3909 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:42.299555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b221c3-9c1c-4209-8e0c-871f43ee3909 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:42.299555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b221c3-9c1c-4209-8e0c-871f43ee3909 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:42.301053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 523/1042] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:42.306762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06122f37-e4d5-4797-957a-22b9308036f1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:42.307489 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06122f37-e4d5-4797-957a-22b9308036f1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:42.307489 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06122f37-e4d5-4797-957a-22b9308036f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:42.335224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-06122f37-e4d5-4797-957a-22b9308036f1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:42.336473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-06122f37-e4d5-4797-957a-22b9308036f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:42.449633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 520/1043] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:42 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:42.764700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b221c3-9c1c-4209-8e0c-871f43ee3909 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:42.852879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10305bc8-d73c-4de9-90d9-38ebeefc545e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:42.853351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10305bc8-d73c-4de9-90d9-38ebeefc545e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:42.853930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10305bc8-d73c-4de9-90d9-38ebeefc545e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:42.856666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 521/1044] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:42 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:42.863311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db31aa-5d6c-4a36-9d19-12dae665a7cb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:42.864387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db31aa-5d6c-4a36-9d19-12dae665a7cb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:42.864648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db31aa-5d6c-4a36-9d19-12dae665a7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:42.886115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-21db31aa-5d6c-4a36-9d19-12dae665a7cb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:42.887461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-21db31aa-5d6c-4a36-9d19-12dae665a7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:42.946680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b221c3-9c1c-4209-8e0c-871f43ee3909 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:42.986588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 522/1045] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:42 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:43.002317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4c738-03dd-4548-88d2-b4d7514721c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:43.002986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4c738-03dd-4548-88d2-b4d7514721c6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:43.003311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4c738-03dd-4548-88d2-b4d7514721c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:43.003580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4c738-03dd-4548-88d2-b4d7514721c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:43.004750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4c738-03dd-4548-88d2-b4d7514721c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:43.006622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4c738-03dd-4548-88d2-b4d7514721c6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:43.008955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 524/1046] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:42 2022] GET /identity/v3/domains/default => generated 200 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:43.014962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db31aa-5d6c-4a36-9d19-12dae665a7cb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:43.044178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:43.202461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db31aa-5d6c-4a36-9d19-12dae665a7cb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:43.268320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcb9a-62f1-4d6f-87a4-19fafb57166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:43.268320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcb9a-62f1-4d6f-87a4-19fafb57166b None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:43.268320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcb9a-62f1-4d6f-87a4-19fafb57166b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:43.268320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcb9a-62f1-4d6f-87a4-19fafb57166b None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:43.271595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bcb9a-62f1-4d6f-87a4-19fafb57166b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:43.274452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bcb9a-62f1-4d6f-87a4-19fafb57166b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:43.295757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4c738-03dd-4548-88d2-b4d7514721c6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:43.317905 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 523/1047] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:24:43 2022] POST /identity/v3/users => generated 286 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:24:43.377318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:43.445198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4c738-03dd-4548-88d2-b4d7514721c6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:43.497517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb810d-938d-4ca4-958c-756bc72ebde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:43.498542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb810d-938d-4ca4-958c-756bc72ebde7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:43.499062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb810d-938d-4ca4-958c-756bc72ebde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:43.499567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb810d-938d-4ca4-958c-756bc72ebde7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:43.517048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb810d-938d-4ca4-958c-756bc72ebde7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7cfb810d-938d-4ca4-958c-756bc72ebde7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:43.522813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:43.524753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:43.526546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 525/1048] 10.222.0.100 () {56 vars in 1125 bytes} [Thu Sep 1 17:24:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:43.530760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcb9a-62f1-4d6f-87a4-19fafb57166b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:43.669810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:43.727424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcb9a-62f1-4d6f-87a4-19fafb57166b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:43.790450 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3427af-e5c4-48fe-b2d6-51bfe244736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:43.791415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3427af-e5c4-48fe-b2d6-51bfe244736c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:43.791921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3427af-e5c4-48fe-b2d6-51bfe244736c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:43.792415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3427af-e5c4-48fe-b2d6-51bfe244736c None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:43.793704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3427af-e5c4-48fe-b2d6-51bfe244736c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:43.794261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3427af-e5c4-48fe-b2d6-51bfe244736c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:43.796917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3427af-e5c4-48fe-b2d6-51bfe244736c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:43.817262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 524/1049] 10.222.0.100 () {56 vars in 1120 bytes} [Thu Sep 1 17:24:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:43.819407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb810d-938d-4ca4-958c-756bc72ebde7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:43.969877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb810d-938d-4ca4-958c-756bc72ebde7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:43.995904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:44.018190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06815f8d-58bb-4412-9587-b5c7013a48a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:44.018839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06815f8d-58bb-4412-9587-b5c7013a48a0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.019122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06815f8d-58bb-4412-9587-b5c7013a48a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.019398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06815f8d-58bb-4412-9587-b5c7013a48a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.032091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06815f8d-58bb-4412-9587-b5c7013a48a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:44.033542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06815f8d-58bb-4412-9587-b5c7013a48a0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-06815f8d-58bb-4412-9587-b5c7013a48a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:44.056740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:44.057962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:44.059228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:44.059228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:44.059228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:44.059228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 526/1050] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:44.065295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3427af-e5c4-48fe-b2d6-51bfe244736c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:44.122407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c329da-004e-469a-a86f-bf9fd0f0b3a9 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.122773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c329da-004e-469a-a86f-bf9fd0f0b3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.123197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c329da-004e-469a-a86f-bf9fd0f0b3a9 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.124430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 527/1051] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:44.129922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d03a7-ace7-4913-9f9c-18d315d5a113 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.130313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d03a7-ace7-4913-9f9c-18d315d5a113 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.130764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d03a7-ace7-4913-9f9c-18d315d5a113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.150895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e28d03a7-ace7-4913-9f9c-18d315d5a113 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:44.152204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e28d03a7-ace7-4913-9f9c-18d315d5a113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:44.179020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3427af-e5c4-48fe-b2d6-51bfe244736c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:44.220730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbabee7-fa13-474d-aa57-4b43e0553586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:44.221370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbabee7-fa13-474d-aa57-4b43e0553586 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.221675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbabee7-fa13-474d-aa57-4b43e0553586 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.221949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbabee7-fa13-474d-aa57-4b43e0553586 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.222593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbabee7-fa13-474d-aa57-4b43e0553586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:44.222892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbabee7-fa13-474d-aa57-4b43e0553586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:44.224962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbabee7-fa13-474d-aa57-4b43e0553586 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:44.226473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:44.226980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 528/1052] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:44 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:44.249822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 525/1053] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:44.251082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d03a7-ace7-4913-9f9c-18d315d5a113 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:44.357018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:44.366043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d03a7-ace7-4913-9f9c-18d315d5a113 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:44.413640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31448cdb-eb6f-4d2d-9e2e-2ec0ea0a37ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:44.414987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31448cdb-eb6f-4d2d-9e2e-2ec0ea0a37ee None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.415267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31448cdb-eb6f-4d2d-9e2e-2ec0ea0a37ee None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.415533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31448cdb-eb6f-4d2d-9e2e-2ec0ea0a37ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.424182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31448cdb-eb6f-4d2d-9e2e-2ec0ea0a37ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:44.426261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31448cdb-eb6f-4d2d-9e2e-2ec0ea0a37ee None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-31448cdb-eb6f-4d2d-9e2e-2ec0ea0a37ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:44.441682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:44.443170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:44.444772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:44.445743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 529/1054] 10.222.0.100 () {56 vars in 1098 bytes} [Thu Sep 1 17:24:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:44.448984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbabee7-fa13-474d-aa57-4b43e0553586 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:44.497955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31448cdb-eb6f-4d2d-9e2e-2ec0ea0a37ee None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:44.568568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbabee7-fa13-474d-aa57-4b43e0553586 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:44.616461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced8ac8-8d1d-42a7-8577-4a3f2de93f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:44.617332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced8ac8-8d1d-42a7-8577-4a3f2de93f78 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.617648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced8ac8-8d1d-42a7-8577-4a3f2de93f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.617925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced8ac8-8d1d-42a7-8577-4a3f2de93f78 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.618671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced8ac8-8d1d-42a7-8577-4a3f2de93f78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:44.618976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced8ac8-8d1d-42a7-8577-4a3f2de93f78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:44.620417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced8ac8-8d1d-42a7-8577-4a3f2de93f78 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:44.627567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31448cdb-eb6f-4d2d-9e2e-2ec0ea0a37ee None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:44.646023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 526/1055] 10.222.0.100 () {56 vars in 1138 bytes} [Thu Sep 1 17:24:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:44.647889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced8ac8-8d1d-42a7-8577-4a3f2de93f78 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:44.684082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a22a6-99c5-40c8-9964-d7ad3139fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:44.685565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a22a6-99c5-40c8-9964-d7ad3139fabd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.686255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a22a6-99c5-40c8-9964-d7ad3139fabd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.686812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a22a6-99c5-40c8-9964-d7ad3139fabd None admin] PATH_INFO: `/v3/services/image` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.687611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a22a6-99c5-40c8-9964-d7ad3139fabd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:44.690107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a22a6-99c5-40c8-9964-d7ad3139fabd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9d1a22a6-99c5-40c8-9964-d7ad3139fabd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:44.699776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:44.701041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:44.702463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 530/1056] 10.222.0.100 () {56 vars in 1110 bytes} [Thu Sep 1 17:24:44 2022] GET /identity/v3/services/image => generated 86 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:44.707553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a22a6-99c5-40c8-9964-d7ad3139fabd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:44.717030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:44.734372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:44.773456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced8ac8-8d1d-42a7-8577-4a3f2de93f78 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:44.816618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a055c-797a-4d76-94a1-63183625a440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:44.817260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a055c-797a-4d76-94a1-63183625a440 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.817537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a055c-797a-4d76-94a1-63183625a440 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.817875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a055c-797a-4d76-94a1-63183625a440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.829875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a055c-797a-4d76-94a1-63183625a440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:44.831327 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a055c-797a-4d76-94a1-63183625a440 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:44.831612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a22a6-99c5-40c8-9964-d7ad3139fabd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5c7a055c-797a-4d76-94a1-63183625a440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:44.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:44.848886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:44.850726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:44.850726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:44.850726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:44.850726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 527/1057] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:44.855530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a055c-797a-4d76-94a1-63183625a440 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:44.874431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd132c-ade2-4706-b086-9ff5ffdee567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:44.874962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd132c-ade2-4706-b086-9ff5ffdee567 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:44.875251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd132c-ade2-4706-b086-9ff5ffdee567 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:44.875529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd132c-ade2-4706-b086-9ff5ffdee567 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:44.876176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd132c-ade2-4706-b086-9ff5ffdee567 None admin] RBAC: Adding query filter params (name=image) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:44.876478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd132c-ade2-4706-b086-9ff5ffdee567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:44.877906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd132c-ade2-4706-b086-9ff5ffdee567 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:44.884708 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 531/1058] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:44 2022] GET /identity/v3/services?name=image => generated 123 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:44.890532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd132c-ade2-4706-b086-9ff5ffdee567 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:44.979496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a055c-797a-4d76-94a1-63183625a440 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:44.982342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:44.990395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:45.006774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd132c-ade2-4706-b086-9ff5ffdee567 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:45.027692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a6994-89a6-409b-80f4-59512135c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:45.028303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a6994-89a6-409b-80f4-59512135c1db None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.028843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a6994-89a6-409b-80f4-59512135c1db None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.029110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a6994-89a6-409b-80f4-59512135c1db None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:45.029893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a6994-89a6-409b-80f4-59512135c1db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:45.030195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a6994-89a6-409b-80f4-59512135c1db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:45.031559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a6994-89a6-409b-80f4-59512135c1db None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:45.049743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 528/1059] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:45.050932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a6994-89a6-409b-80f4-59512135c1db None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:45.051774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fda43f-c7c1-47da-9483-e89f57d192ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:45.052408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fda43f-c7c1-47da-9483-e89f57d192ef None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.052694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fda43f-c7c1-47da-9483-e89f57d192ef None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.053026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fda43f-c7c1-47da-9483-e89f57d192ef None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:45.053742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fda43f-c7c1-47da-9483-e89f57d192ef None admin] RBAC: Adding query filter params (type=image) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:45.053932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fda43f-c7c1-47da-9483-e89f57d192ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:45.055327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fda43f-c7c1-47da-9483-e89f57d192ef None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:45.061306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 532/1060] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:44 2022] GET /identity/v3/services?type=image => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:45.067107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fda43f-c7c1-47da-9483-e89f57d192ef None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:45.185040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fda43f-c7c1-47da-9483-e89f57d192ef None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:45.192150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a6994-89a6-409b-80f4-59512135c1db None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:45.232229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5d277-7ebe-47fe-b1f6-7e8f885c9e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:45.232842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d277-7ebe-47fe-b1f6-7e8f885c9e7c None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.233130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d277-7ebe-47fe-b1f6-7e8f885c9e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.233410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d277-7ebe-47fe-b1f6-7e8f885c9e7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:45.234111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5d277-7ebe-47fe-b1f6-7e8f885c9e7c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:45.234987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cf4af-5d23-47d9-b27e-15cfccf91c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:45.235480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5d277-7ebe-47fe-b1f6-7e8f885c9e7c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:45.235942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cf4af-5d23-47d9-b27e-15cfccf91c36 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.236369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cf4af-5d23-47d9-b27e-15cfccf91c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.236802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cf4af-5d23-47d9-b27e-15cfccf91c36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:45.244670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cf4af-5d23-47d9-b27e-15cfccf91c36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:45.246172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cf4af-5d23-47d9-b27e-15cfccf91c36 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:45.265444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 533/1061] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:24:45 2022] POST /identity/v3/endpoints => generated 329 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4c2cf4af-5d23-47d9-b27e-15cfccf91c36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:45.265920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:45.266861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:45.267779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:45.267779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:45.267779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:45.267779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:45.267779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:45.267779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:45.267779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:45.267779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:45.267779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 529/1062] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:45.268306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5d277-7ebe-47fe-b1f6-7e8f885c9e7c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:45.275416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:45.411111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f44e4-3694-483c-ab3b-3139bfc975f9 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.412378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f44e4-3694-483c-ab3b-3139bfc975f9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.412541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5d277-7ebe-47fe-b1f6-7e8f885c9e7c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:45.412650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f44e4-3694-483c-ab3b-3139bfc975f9 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:45.414009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 530/1063] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:45.419319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2b803-a962-4c90-a33c-b611e2c0a4ab None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.419610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2b803-a962-4c90-a33c-b611e2c0a4ab None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.419889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2b803-a962-4c90-a33c-b611e2c0a4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:45.444858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-78f2b803-a962-4c90-a33c-b611e2c0a4ab None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:45.446273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-78f2b803-a962-4c90-a33c-b611e2c0a4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:45.451393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c44db-d42b-4074-9a9b-f6390462aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:45.452048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c44db-d42b-4074-9a9b-f6390462aa85 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.452276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c44db-d42b-4074-9a9b-f6390462aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.452542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c44db-d42b-4074-9a9b-f6390462aa85 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:45.453248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c44db-d42b-4074-9a9b-f6390462aa85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:45.453549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c44db-d42b-4074-9a9b-f6390462aa85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:45.455009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c44db-d42b-4074-9a9b-f6390462aa85 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:45.467754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:45.470443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 534/1064] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:45.481613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c44db-d42b-4074-9a9b-f6390462aa85 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:45.551937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:45.563078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 531/1065] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:45 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:45.608692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c44db-d42b-4074-9a9b-f6390462aa85 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:45.653725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e1c3a-4ce2-4a17-a548-54d33ea050c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:45.653725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e1c3a-4ce2-4a17-a548-54d33ea050c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.653725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e1c3a-4ce2-4a17-a548-54d33ea050c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.653725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e1c3a-4ce2-4a17-a548-54d33ea050c2 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/c34f6f3e93b54a5d92b6f2f9f0de40f6/roles/42a25be79662491da6d022c8d6fb860f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:45.656695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e1c3a-4ce2-4a17-a548-54d33ea050c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=c34f6f3e93b54a5d92b6f2f9f0de40f6, role_id=42a25be79662491da6d022c8d6fb860f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:45.659090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e1c3a-4ce2-4a17-a548-54d33ea050c2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:45.693898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 535/1066] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:24:45 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/c34f6f3e93b54a5d92b6f2f9f0de40f6/roles/42a25be79662491da6d022c8d6fb860f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:24:45.846978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2b803-a962-4c90-a33c-b611e2c0a4ab None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:45.959056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2b803-a962-4c90-a33c-b611e2c0a4ab None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:45.998260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdde705-bffa-434e-858c-114e73f435ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:45.998897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdde705-bffa-434e-858c-114e73f435ed None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:45.999174 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdde705-bffa-434e-858c-114e73f435ed None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:45.999439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdde705-bffa-434e-858c-114e73f435ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:46.012137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdde705-bffa-434e-858c-114e73f435ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2bdde705-bffa-434e-858c-114e73f435ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:46.017818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:46.019590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:46.020901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 532/1067] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:46.026089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e1c3a-4ce2-4a17-a548-54d33ea050c2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:46.165391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e1c3a-4ce2-4a17-a548-54d33ea050c2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:46.190206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:46.207270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc9b05-48d1-4ea4-b7d5-67f6d08c458f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:46.207886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc9b05-48d1-4ea4-b7d5-67f6d08c458f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:46.208164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc9b05-48d1-4ea4-b7d5-67f6d08c458f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:46.208582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc9b05-48d1-4ea4-b7d5-67f6d08c458f None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:46.209847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc9b05-48d1-4ea4-b7d5-67f6d08c458f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:46.210156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc9b05-48d1-4ea4-b7d5-67f6d08c458f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:46.211541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc9b05-48d1-4ea4-b7d5-67f6d08c458f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:46.220968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 536/1068] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:24:46 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:46.227382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdde705-bffa-434e-858c-114e73f435ed None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:46.245339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:46.344620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdde705-bffa-434e-858c-114e73f435ed None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:46.392462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e036c-e0c1-4a14-be2f-42888de7fe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:46.393101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e036c-e0c1-4a14-be2f-42888de7fe08 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:46.393377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e036c-e0c1-4a14-be2f-42888de7fe08 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:46.393681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e036c-e0c1-4a14-be2f-42888de7fe08 None admin] PATH_INFO: `/v3/users/218deefaf50644e9ba17c7a0f78ec39c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:46.409973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e036c-e0c1-4a14-be2f-42888de7fe08 None admin] RBAC: Authorizing `identity:get_user(user_id=218deefaf50644e9ba17c7a0f78ec39c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:46.411434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e036c-e0c1-4a14-be2f-42888de7fe08 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:46.414619 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 533/1069] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:24:46 2022] GET /identity/v3/users/218deefaf50644e9ba17c7a0f78ec39c => generated 286 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:46.419803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc9b05-48d1-4ea4-b7d5-67f6d08c458f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:46.539434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc9b05-48d1-4ea4-b7d5-67f6d08c458f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:46.579279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f2fd4-9c6b-41cf-a264-5af2d6883cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:46.579946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f2fd4-9c6b-41cf-a264-5af2d6883cbb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:46.580229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f2fd4-9c6b-41cf-a264-5af2d6883cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:46.580502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f2fd4-9c6b-41cf-a264-5af2d6883cbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:46.648752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:46.681955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f2fd4-9c6b-41cf-a264-5af2d6883cbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:46.683330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f2fd4-9c6b-41cf-a264-5af2d6883cbb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:46.808397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 537/1070] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:46 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:24:46.813887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e036c-e0c1-4a14-be2f-42888de7fe08 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:46.901141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:46.955512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e036c-e0c1-4a14-be2f-42888de7fe08 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:47.023938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc3257-04a1-446c-ab21-34ad32224a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:47.025037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc3257-04a1-446c-ab21-34ad32224a0c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:47.025525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc3257-04a1-446c-ab21-34ad32224a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:47.026050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc3257-04a1-446c-ab21-34ad32224a0c None admin] PATH_INFO: `/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:47.028346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc3257-04a1-446c-ab21-34ad32224a0c None admin] RBAC: Authorizing `identity:get_project(project_id=6f040600c7e14e0a8f0ecc791c9266fe)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:47.031442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc3257-04a1-446c-ab21-34ad32224a0c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:47.036245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 534/1071] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:24:46 2022] GET /identity/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:47.045562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f2fd4-9c6b-41cf-a264-5af2d6883cbb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:47.154008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:47.178094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f2fd4-9c6b-41cf-a264-5af2d6883cbb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:47.233400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7ff36-5b5c-4197-99bf-1cf1121657a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:47.233937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7ff36-5b5c-4197-99bf-1cf1121657a5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:47.234217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7ff36-5b5c-4197-99bf-1cf1121657a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:47.234488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7ff36-5b5c-4197-99bf-1cf1121657a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:47.235232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7ff36-5b5c-4197-99bf-1cf1121657a5 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=6f040600c7e14e0a8f0ecc791c9266fe, user.id=218deefaf50644e9ba17c7a0f78ec39c) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:47.235539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7ff36-5b5c-4197-99bf-1cf1121657a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:47.236995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7ff36-5b5c-4197-99bf-1cf1121657a5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:47.249542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 538/1072] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:47 2022] GET /identity/v3/role_assignments?user.id=218deefaf50644e9ba17c7a0f78ec39c&scope.project.id=6f040600c7e14e0a8f0ecc791c9266fe&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:47.318897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72237fbe-edc3-4f75-9f7d-a6d1f62892f8 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:47.319207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72237fbe-edc3-4f75-9f7d-a6d1f62892f8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:47.319431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72237fbe-edc3-4f75-9f7d-a6d1f62892f8 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:47.320852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 535/1073] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:47.326817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390d46a-da6e-4644-9f44-f203f653b85b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:47.327197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390d46a-da6e-4644-9f44-f203f653b85b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:47.327551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390d46a-da6e-4644-9f44-f203f653b85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:47.348678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e390d46a-da6e-4644-9f44-f203f653b85b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:47.350927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e390d46a-da6e-4644-9f44-f203f653b85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:47.437999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 539/1074] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:47 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:47.708356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72237fbe-edc3-4f75-9f7d-a6d1f62892f8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:47.863798 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72237fbe-edc3-4f75-9f7d-a6d1f62892f8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:47.907319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510cf23-252d-4252-8d6b-7bfd4f01560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:47.908037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510cf23-252d-4252-8d6b-7bfd4f01560b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:47.908599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510cf23-252d-4252-8d6b-7bfd4f01560b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:47.909004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510cf23-252d-4252-8d6b-7bfd4f01560b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:47.929273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:48.020349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510cf23-252d-4252-8d6b-7bfd4f01560b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:48.023237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510cf23-252d-4252-8d6b-7bfd4f01560b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:48.099986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2a3c0-8a25-43bd-857b-f4afabf6de9f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:48.100987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2a3c0-8a25-43bd-857b-f4afabf6de9f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:48.100987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2a3c0-8a25-43bd-857b-f4afabf6de9f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:48.102631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 540/1075] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:48.108416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab190ef8-d31e-476a-a990-8b3546906ab0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:48.108669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab190ef8-d31e-476a-a990-8b3546906ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:48.109006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab190ef8-d31e-476a-a990-8b3546906ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:48.128158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ab190ef8-d31e-476a-a990-8b3546906ab0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:48.129382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ab190ef8-d31e-476a-a990-8b3546906ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:48.215368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 536/1076] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:47 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:24:48.221931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 541/1077] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:48 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:48.225181 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510cf23-252d-4252-8d6b-7bfd4f01560b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:48.312829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:48.387093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510cf23-252d-4252-8d6b-7bfd4f01560b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:48.426184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6cb2d-9abf-4b04-aaad-3b422d709c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:48.426833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6cb2d-9abf-4b04-aaad-3b422d709c82 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:48.427124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6cb2d-9abf-4b04-aaad-3b422d709c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:48.428023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6cb2d-9abf-4b04-aaad-3b422d709c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:48.439494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6cb2d-9abf-4b04-aaad-3b422d709c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:48.441029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6cb2d-9abf-4b04-aaad-3b422d709c82 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c8d6cb2d-9abf-4b04-aaad-3b422d709c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:48.457675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:48.459120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:48.460052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:48.460052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:48.460052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:48.460052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 537/1078] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:48.467349 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab190ef8-d31e-476a-a990-8b3546906ab0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:48.481235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:48.500290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6cb2d-9abf-4b04-aaad-3b422d709c82 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:48.612499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab190ef8-d31e-476a-a990-8b3546906ab0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:48.660101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:48.674633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe9314-f272-46b4-bcac-d662d2c6f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:48.675587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe9314-f272-46b4-bcac-d662d2c6f538 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:48.676030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe9314-f272-46b4-bcac-d662d2c6f538 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:48.676455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe9314-f272-46b4-bcac-d662d2c6f538 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:48.677298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe9314-f272-46b4-bcac-d662d2c6f538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:48.677824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe9314-f272-46b4-bcac-d662d2c6f538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:48.679447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe9314-f272-46b4-bcac-d662d2c6f538 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:48.680161 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6cb2d-9abf-4b04-aaad-3b422d709c82 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:48.695120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 542/1079] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:48.743097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0f608-e23e-44d6-b010-54a600c07452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:48.744272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0f608-e23e-44d6-b010-54a600c07452 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:48.744822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0f608-e23e-44d6-b010-54a600c07452 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:48.745320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0f608-e23e-44d6-b010-54a600c07452 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:48.768668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0f608-e23e-44d6-b010-54a600c07452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a1b0f608-e23e-44d6-b010-54a600c07452 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:48.779497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:48.781400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:48.788138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 538/1080] 10.222.0.100 () {56 vars in 1125 bytes} [Thu Sep 1 17:24:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:48.794409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe9314-f272-46b4-bcac-d662d2c6f538 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:48.904735 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:48.951788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe9314-f272-46b4-bcac-d662d2c6f538 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:49.000445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecc740-4e41-49a2-8c48-7926aebc1b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:49.001280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecc740-4e41-49a2-8c48-7926aebc1b65 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:49.001757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecc740-4e41-49a2-8c48-7926aebc1b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:49.002081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecc740-4e41-49a2-8c48-7926aebc1b65 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:49.003104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecc740-4e41-49a2-8c48-7926aebc1b65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:49.003607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecc740-4e41-49a2-8c48-7926aebc1b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:49.007342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecc740-4e41-49a2-8c48-7926aebc1b65 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:49.024840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 543/1081] 10.222.0.100 () {56 vars in 1120 bytes} [Thu Sep 1 17:24:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:49.031076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0f608-e23e-44d6-b010-54a600c07452 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:49.183315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0f608-e23e-44d6-b010-54a600c07452 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:49.219599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f2cb1-5343-401e-b12e-ab4d95856869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:49.220251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f2cb1-5343-401e-b12e-ab4d95856869 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:49.220532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f2cb1-5343-401e-b12e-ab4d95856869 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:49.220805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f2cb1-5343-401e-b12e-ab4d95856869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:49.235880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f2cb1-5343-401e-b12e-ab4d95856869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:49.238697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f2cb1-5343-401e-b12e-ab4d95856869 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8a2f2cb1-5343-401e-b12e-ab4d95856869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:49.262256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:49.264313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:49.265594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:49.265594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:49.265594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:49.265594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 539/1082] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:49.275031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecc740-4e41-49a2-8c48-7926aebc1b65 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:49.431748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecc740-4e41-49a2-8c48-7926aebc1b65 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:49.465480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:49.476207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f233216-c7fc-42db-b051-558515910f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:49.476851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f233216-c7fc-42db-b051-558515910f9d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:49.477125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f233216-c7fc-42db-b051-558515910f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:49.477390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f233216-c7fc-42db-b051-558515910f9d None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:49.478115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f233216-c7fc-42db-b051-558515910f9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:49.478416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f233216-c7fc-42db-b051-558515910f9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:49.479863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f233216-c7fc-42db-b051-558515910f9d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:49.494327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 544/1083] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:49.501024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f2cb1-5343-401e-b12e-ab4d95856869 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:49.507453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:49.570068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46856bc9-cdf1-430f-8b6e-2847f8f866d0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:49.570264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46856bc9-cdf1-430f-8b6e-2847f8f866d0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:49.570361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46856bc9-cdf1-430f-8b6e-2847f8f866d0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:49.571772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 545/1084] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:24:49.577889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066ee02-19db-4c6d-8869-50af117bc0c0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:49.578169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066ee02-19db-4c6d-8869-50af117bc0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:49.578440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066ee02-19db-4c6d-8869-50af117bc0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:49.597991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c066ee02-19db-4c6d-8869-50af117bc0c0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:49.599897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c066ee02-19db-4c6d-8869-50af117bc0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:49.668199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f2cb1-5343-401e-b12e-ab4d95856869 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:49.696373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 546/1085] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:49 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:24:49.728092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddce855-ea56-416d-a8af-036ab4b00314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:49.729167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddce855-ea56-416d-a8af-036ab4b00314 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:49.729681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddce855-ea56-416d-a8af-036ab4b00314 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:49.730157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddce855-ea56-416d-a8af-036ab4b00314 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:49.743981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddce855-ea56-416d-a8af-036ab4b00314 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:49.745454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddce855-ea56-416d-a8af-036ab4b00314 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-6ddce855-ea56-416d-a8af-036ab4b00314 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:24:49.759566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:49.760651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:49.761643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 540/1086] 10.222.0.100 () {56 vars in 1098 bytes} [Thu Sep 1 17:24:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:49.767584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066ee02-19db-4c6d-8869-50af117bc0c0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:49.906272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066ee02-19db-4c6d-8869-50af117bc0c0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:49.940565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:49.965378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c9ea1-304d-4ecf-9bf9-ed7ef4605f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:49.966026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c9ea1-304d-4ecf-9bf9-ed7ef4605f30 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:49.966314 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c9ea1-304d-4ecf-9bf9-ed7ef4605f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:49.966591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c9ea1-304d-4ecf-9bf9-ed7ef4605f30 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:49.967343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c9ea1-304d-4ecf-9bf9-ed7ef4605f30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:49.967643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c9ea1-304d-4ecf-9bf9-ed7ef4605f30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:49.969116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c9ea1-304d-4ecf-9bf9-ed7ef4605f30 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:49.994704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddce855-ea56-416d-a8af-036ab4b00314 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:49.995412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 547/1087] 10.222.0.100 () {56 vars in 1138 bytes} [Thu Sep 1 17:24:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:50.001420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c9ea1-304d-4ecf-9bf9-ed7ef4605f30 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:50.087934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:50.134625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddce855-ea56-416d-a8af-036ab4b00314 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.137162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c9ea1-304d-4ecf-9bf9-ed7ef4605f30 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.172831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038010b-5479-4fa9-bd80-3f7805ebd309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.173390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038010b-5479-4fa9-bd80-3f7805ebd309 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.173390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038010b-5479-4fa9-bd80-3f7805ebd309 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.173852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038010b-5479-4fa9-bd80-3f7805ebd309 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.179830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb3e44-bdd0-4aa7-be7c-337cdca13fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.180449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb3e44-bdd0-4aa7-be7c-337cdca13fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.180726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb3e44-bdd0-4aa7-be7c-337cdca13fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.180994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb3e44-bdd0-4aa7-be7c-337cdca13fc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.186148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038010b-5479-4fa9-bd80-3f7805ebd309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-6038010b-5479-4fa9-bd80-3f7805ebd309 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:50.191092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:50.192291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:50.194044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 541/1088] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:50.198595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038010b-5479-4fa9-bd80-3f7805ebd309 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:50.240004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:50.265692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb3e44-bdd0-4aa7-be7c-337cdca13fc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:50.267149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb3e44-bdd0-4aa7-be7c-337cdca13fc7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:50.320081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038010b-5479-4fa9-bd80-3f7805ebd309 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.345996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:50.363482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1f474-e800-45d4-8a83-4efa6326a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.364097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1f474-e800-45d4-8a83-4efa6326a857 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.364368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1f474-e800-45d4-8a83-4efa6326a857 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.364752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1f474-e800-45d4-8a83-4efa6326a857 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.365409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1f474-e800-45d4-8a83-4efa6326a857 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:50.365822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1f474-e800-45d4-8a83-4efa6326a857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:50.367189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1f474-e800-45d4-8a83-4efa6326a857 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:50.378422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 542/1089] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:24:50 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:50.387816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1f474-e800-45d4-8a83-4efa6326a857 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:50.391509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 548/1090] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:49 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:24:50.399648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb3e44-bdd0-4aa7-be7c-337cdca13fc7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:50.492565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:50.518018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb3e44-bdd0-4aa7-be7c-337cdca13fc7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.537511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1f474-e800-45d4-8a83-4efa6326a857 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.562606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1afb5-c621-4326-9a4e-9cf112ddd0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.563235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1afb5-c621-4326-9a4e-9cf112ddd0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.563509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1afb5-c621-4326-9a4e-9cf112ddd0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.563776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1afb5-c621-4326-9a4e-9cf112ddd0c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.575526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1afb5-c621-4326-9a4e-9cf112ddd0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:50.576991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1afb5-c621-4326-9a4e-9cf112ddd0c2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:50.577779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea288d-802c-41cb-aa2f-ab051aecc5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.578473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea288d-802c-41cb-aa2f-ab051aecc5be None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.578932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea288d-802c-41cb-aa2f-ab051aecc5be None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.579250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea288d-802c-41cb-aa2f-ab051aecc5be None admin] PATH_INFO: `/v3/users/218deefaf50644e9ba17c7a0f78ec39c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.580652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea288d-802c-41cb-aa2f-ab051aecc5be None admin] RBAC: Authorizing `identity:get_user(user_id=218deefaf50644e9ba17c7a0f78ec39c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:50.582182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea288d-802c-41cb-aa2f-ab051aecc5be None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:50.584678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 543/1091] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:24:50 2022] GET /identity/v3/users/218deefaf50644e9ba17c7a0f78ec39c => generated 286 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:50.590762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea288d-802c-41cb-aa2f-ab051aecc5be None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6ba1afb5-c621-4326-9a4e-9cf112ddd0c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:24:50.594299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:50.595370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:24:50.597061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:24:50.597061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:24:50.597061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:50.597061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 549/1092] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:50.601437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1afb5-c621-4326-9a4e-9cf112ddd0c2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:50.665300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:50.718486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea288d-802c-41cb-aa2f-ab051aecc5be None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.721778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1afb5-c621-4326-9a4e-9cf112ddd0c2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.753698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:50.764721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad632fe-2f59-48af-b361-7ef8680ff232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.765371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad632fe-2f59-48af-b361-7ef8680ff232 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.765851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad632fe-2f59-48af-b361-7ef8680ff232 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.766161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad632fe-2f59-48af-b361-7ef8680ff232 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.766922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad632fe-2f59-48af-b361-7ef8680ff232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:50.767273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad632fe-2f59-48af-b361-7ef8680ff232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:50.768703 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c331c-def0-4750-a070-911fb6daa650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.768831 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad632fe-2f59-48af-b361-7ef8680ff232 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:50.769342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c331c-def0-4750-a070-911fb6daa650 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.769642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c331c-def0-4750-a070-911fb6daa650 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.769917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c331c-def0-4750-a070-911fb6daa650 None admin] PATH_INFO: `/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.771139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c331c-def0-4750-a070-911fb6daa650 None admin] RBAC: Authorizing `identity:get_project(project_id=6f040600c7e14e0a8f0ecc791c9266fe)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:50.772678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c331c-def0-4750-a070-911fb6daa650 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:50.774580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 544/1093] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:24:50 2022] GET /identity/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:50.782355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c331c-def0-4750-a070-911fb6daa650 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:50.785758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 550/1094] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:24:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:50.795265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad632fe-2f59-48af-b361-7ef8680ff232 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:50.911418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c331c-def0-4750-a070-911fb6daa650 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.920847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad632fe-2f59-48af-b361-7ef8680ff232 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:50.937623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:50.954005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd953151-fcfe-40eb-aa86-ca6ec2a03381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.954676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd953151-fcfe-40eb-aa86-ca6ec2a03381 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.954947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd953151-fcfe-40eb-aa86-ca6ec2a03381 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.955215 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd953151-fcfe-40eb-aa86-ca6ec2a03381 None admin] PATH_INFO: `/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe/users/218deefaf50644e9ba17c7a0f78ec39c/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.957234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd953151-fcfe-40eb-aa86-ca6ec2a03381 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f040600c7e14e0a8f0ecc791c9266fe, user_id=218deefaf50644e9ba17c7a0f78ec39c, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:50.959420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd953151-fcfe-40eb-aa86-ca6ec2a03381 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:50.969275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cedd0-fdaf-4283-a3ba-facaf00eb8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:50.969955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cedd0-fdaf-4283-a3ba-facaf00eb8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:50.970235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cedd0-fdaf-4283-a3ba-facaf00eb8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:50.970501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cedd0-fdaf-4283-a3ba-facaf00eb8a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:50.974938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 545/1095] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:24:50 2022] PUT /identity/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe/users/218deefaf50644e9ba17c7a0f78ec39c/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:24:50.977871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cedd0-fdaf-4283-a3ba-facaf00eb8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:50.979364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cedd0-fdaf-4283-a3ba-facaf00eb8a2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b21cedd0-fdaf-4283-a3ba-facaf00eb8a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:24:50.993150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:50.994682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:50.995968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:24:50.995968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:24:50.995968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:24:50.995968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:24:50.995968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:24:50.995968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:24:50.995968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:24:50.995968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:24:50.995968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 551/1096] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:24:51.001178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd953151-fcfe-40eb-aa86-ca6ec2a03381 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:51.162795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd953151-fcfe-40eb-aa86-ca6ec2a03381 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:51.200130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde85a6-87a7-4be5-8798-56024d4da312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:51.200745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde85a6-87a7-4be5-8798-56024d4da312 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:51.201113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde85a6-87a7-4be5-8798-56024d4da312 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:51.201277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde85a6-87a7-4be5-8798-56024d4da312 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:51.202008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde85a6-87a7-4be5-8798-56024d4da312 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:51.202316 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde85a6-87a7-4be5-8798-56024d4da312 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:51.203757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde85a6-87a7-4be5-8798-56024d4da312 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:51.217929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 546/1097] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:24:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:51.226112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cedd0-fdaf-4283-a3ba-facaf00eb8a2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:51.295618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:51.353528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cedd0-fdaf-4283-a3ba-facaf00eb8a2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:51.394186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf20733-d72f-4ac8-b3d1-34f54e805f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:51.394834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf20733-d72f-4ac8-b3d1-34f54e805f8a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:51.395058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf20733-d72f-4ac8-b3d1-34f54e805f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:51.395325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf20733-d72f-4ac8-b3d1-34f54e805f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:51.396603 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf20733-d72f-4ac8-b3d1-34f54e805f8a None admin] RBAC: Adding query filter params (role.id=42a25be79662491da6d022c8d6fb860f, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:51.396919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf20733-d72f-4ac8-b3d1-34f54e805f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:51.398411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf20733-d72f-4ac8-b3d1-34f54e805f8a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:51.405500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 552/1098] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:51 2022] GET /identity/v3/role_assignments?user.id=c34f6f3e93b54a5d92b6f2f9f0de40f6&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=42a25be79662491da6d022c8d6fb860f => generated 610 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:53.011872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0ad97-1566-4242-8ed1-0a7e3c139752 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:53.012394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0ad97-1566-4242-8ed1-0a7e3c139752 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:53.012394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0ad97-1566-4242-8ed1-0a7e3c139752 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:53.013822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 547/1099] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:53.022260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bab41-ed3d-4251-8996-be298176ecbb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:53.022823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bab41-ed3d-4251-8996-be298176ecbb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:53.023405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bab41-ed3d-4251-8996-be298176ecbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:53.047806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8d5bab41-ed3d-4251-8996-be298176ecbb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:53.049019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8d5bab41-ed3d-4251-8996-be298176ecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:53.125912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 553/1100] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:53 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:53.407848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0ad97-1566-4242-8ed1-0a7e3c139752 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:53.458093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:53.521957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0ad97-1566-4242-8ed1-0a7e3c139752 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:53.558643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97f2c1-cc03-4bbf-8f51-a3aaac27c18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:53.559280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97f2c1-cc03-4bbf-8f51-a3aaac27c18d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:53.559558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97f2c1-cc03-4bbf-8f51-a3aaac27c18d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:53.559828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97f2c1-cc03-4bbf-8f51-a3aaac27c18d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:53.570621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97f2c1-cc03-4bbf-8f51-a3aaac27c18d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-dc97f2c1-cc03-4bbf-8f51-a3aaac27c18d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:24:53.575413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:24:53.576511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:24:53.577556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 548/1101] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:24:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:24:53.582706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bab41-ed3d-4251-8996-be298176ecbb None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:53.628876 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:53.690897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bab41-ed3d-4251-8996-be298176ecbb None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:53.726384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d589ac-8e44-45a6-8ba9-0ed04c0e760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:53.726984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d589ac-8e44-45a6-8ba9-0ed04c0e760f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:53.727261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d589ac-8e44-45a6-8ba9-0ed04c0e760f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:53.727531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d589ac-8e44-45a6-8ba9-0ed04c0e760f None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:53.728195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d589ac-8e44-45a6-8ba9-0ed04c0e760f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:53.728608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d589ac-8e44-45a6-8ba9-0ed04c0e760f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:53.730998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d589ac-8e44-45a6-8ba9-0ed04c0e760f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:53.741374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 554/1102] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:24:53 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:53.747317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97f2c1-cc03-4bbf-8f51-a3aaac27c18d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:53.851354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97f2c1-cc03-4bbf-8f51-a3aaac27c18d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:53.852939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:53.887529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2bdf0-ba95-49c6-99f3-29524b6d0e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:53.888159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2bdf0-ba95-49c6-99f3-29524b6d0e86 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:53.888438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2bdf0-ba95-49c6-99f3-29524b6d0e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:53.888711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2bdf0-ba95-49c6-99f3-29524b6d0e86 None admin] PATH_INFO: `/v3/users/218deefaf50644e9ba17c7a0f78ec39c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:53.889925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2bdf0-ba95-49c6-99f3-29524b6d0e86 None admin] RBAC: Authorizing `identity:get_user(user_id=218deefaf50644e9ba17c7a0f78ec39c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:53.891358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2bdf0-ba95-49c6-99f3-29524b6d0e86 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:53.892947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 549/1103] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:24:53 2022] GET /identity/v3/users/218deefaf50644e9ba17c7a0f78ec39c => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:53.900145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d589ac-8e44-45a6-8ba9-0ed04c0e760f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:54.004912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d589ac-8e44-45a6-8ba9-0ed04c0e760f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:54.007635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:54.040517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134f6d5-0f5f-4560-8602-96d056597f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:54.041202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f6d5-0f5f-4560-8602-96d056597f15 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:54.041390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f6d5-0f5f-4560-8602-96d056597f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:54.041692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f6d5-0f5f-4560-8602-96d056597f15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:54.113810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134f6d5-0f5f-4560-8602-96d056597f15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:54.115377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134f6d5-0f5f-4560-8602-96d056597f15 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:54.253238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 555/1104] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:24:53 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:24:54.260304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2bdf0-ba95-49c6-99f3-29524b6d0e86 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:54.365724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2bdf0-ba95-49c6-99f3-29524b6d0e86 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:54.402575 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f83a4-d224-4ced-973e-f437c1ebde6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:54.403205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f83a4-d224-4ced-973e-f437c1ebde6b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:54.403477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f83a4-d224-4ced-973e-f437c1ebde6b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:54.403998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f83a4-d224-4ced-973e-f437c1ebde6b None admin] PATH_INFO: `/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:54.405117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f83a4-d224-4ced-973e-f437c1ebde6b None admin] RBAC: Authorizing `identity:get_project(project_id=6f040600c7e14e0a8f0ecc791c9266fe)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:54.406577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f83a4-d224-4ced-973e-f437c1ebde6b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:54.408217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 550/1105] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:24:54 2022] GET /identity/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:24:54.414654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134f6d5-0f5f-4560-8602-96d056597f15 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:54.418177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:54.524043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134f6d5-0f5f-4560-8602-96d056597f15 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:54.562483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3dc61-edb5-4591-83e1-32495990edc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:54.563220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3dc61-edb5-4591-83e1-32495990edc2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:54.563497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3dc61-edb5-4591-83e1-32495990edc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:54.563764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3dc61-edb5-4591-83e1-32495990edc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:54.564547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3dc61-edb5-4591-83e1-32495990edc2 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=6f040600c7e14e0a8f0ecc791c9266fe, user.id=218deefaf50644e9ba17c7a0f78ec39c) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:24:54.564850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3dc61-edb5-4591-83e1-32495990edc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:54.566291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3dc61-edb5-4591-83e1-32495990edc2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:54.573094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 556/1106] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:24:54 2022] GET /identity/v3/role_assignments?user.id=218deefaf50644e9ba17c7a0f78ec39c&scope.project.id=6f040600c7e14e0a8f0ecc791c9266fe&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 610 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:56.655436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191dc3-a0a1-4900-bc4b-197064d1bccf None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:56.655967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191dc3-a0a1-4900-bc4b-197064d1bccf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:56.656045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191dc3-a0a1-4900-bc4b-197064d1bccf None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:56.657520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 551/1107] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:56.663975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdeda4-5812-4eeb-8075-bba6c82fca8d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:56.664665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdeda4-5812-4eeb-8075-bba6c82fca8d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:56.664665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdeda4-5812-4eeb-8075-bba6c82fca8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:56.683628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-06fdeda4-5812-4eeb-8075-bba6c82fca8d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:56.684820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-06fdeda4-5812-4eeb-8075-bba6c82fca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:56.749357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:56.755441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 557/1108] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:56 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:57.027597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88191dc3-a0a1-4900-bc4b-197064d1bccf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:57.068027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:57.130673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88191dc3-a0a1-4900-bc4b-197064d1bccf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:57.164697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d92219-86f2-4a2c-9a3b-2d69c400a3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:57.165311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d92219-86f2-4a2c-9a3b-2d69c400a3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:57.165608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d92219-86f2-4a2c-9a3b-2d69c400a3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:57.165883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d92219-86f2-4a2c-9a3b-2d69c400a3c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:57.166984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d92219-86f2-4a2c-9a3b-2d69c400a3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:57.168396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d92219-86f2-4a2c-9a3b-2d69c400a3c9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:57.169986 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 552/1109] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:24:57 2022] GET /identity/v3/domains/default => generated 200 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:57.176069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdeda4-5812-4eeb-8075-bba6c82fca8d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:57.272261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdeda4-5812-4eeb-8075-bba6c82fca8d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:57.305203 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41e9da-116d-4951-8e1d-11f247f72406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:57.305755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41e9da-116d-4951-8e1d-11f247f72406 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:57.306033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41e9da-116d-4951-8e1d-11f247f72406 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:57.306298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41e9da-116d-4951-8e1d-11f247f72406 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:57.306946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41e9da-116d-4951-8e1d-11f247f72406 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:57.308354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41e9da-116d-4951-8e1d-11f247f72406 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:57.330521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 558/1110] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:24:57 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:24:59.299667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c696f-f2f2-4d15-9565-44dc910b10c6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:59.300229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c696f-f2f2-4d15-9565-44dc910b10c6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:59.300229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c696f-f2f2-4d15-9565-44dc910b10c6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:59.301519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 553/1111] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:24:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:24:59.307290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7cea3-1d47-4630-9a33-13f690730968 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:59.307685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7cea3-1d47-4630-9a33-13f690730968 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:59.307825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7cea3-1d47-4630-9a33-13f690730968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:59.326861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f3d7cea3-1d47-4630-9a33-13f690730968 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:24:59.327667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f3d7cea3-1d47-4630-9a33-13f690730968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:24:59.378248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:59.397381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 559/1112] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:24:59 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:24:59.690025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c696f-f2f2-4d15-9565-44dc910b10c6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:59.800227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c696f-f2f2-4d15-9565-44dc910b10c6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:59.808738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:24:59.837598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aba50-71a2-4708-9578-962370f495f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:59.838240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aba50-71a2-4708-9578-962370f495f8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:59.838514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aba50-71a2-4708-9578-962370f495f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:59.838782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aba50-71a2-4708-9578-962370f495f8 None admin] PATH_INFO: `/v3/roles/5dc7056ddc154b81939a17474a64f546` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:59.840157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6aba50-71a2-4708-9578-962370f495f8 None admin] RBAC: Authorizing `identity:get_role(role_id=5dc7056ddc154b81939a17474a64f546)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:24:59.841526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6aba50-71a2-4708-9578-962370f495f8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:24:59.842904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 554/1113] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:24:59 2022] GET /identity/v3/roles/5dc7056ddc154b81939a17474a64f546 => generated 224 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:24:59.848969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7cea3-1d47-4630-9a33-13f690730968 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:24:59.950790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7cea3-1d47-4630-9a33-13f690730968 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:24:59.984267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85dc0-cb66-4c1e-8aac-353c9c8554c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:24:59.984886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85dc0-cb66-4c1e-8aac-353c9c8554c2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:24:59.985158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85dc0-cb66-4c1e-8aac-353c9c8554c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:24:59.985423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85dc0-cb66-4c1e-8aac-353c9c8554c2 None admin] PATH_INFO: `/v3/users/50861fe4d3f142ecbbc04ce92024a0fa` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:24:59.998613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd85dc0-cb66-4c1e-8aac-353c9c8554c2 None admin] RBAC: Authorizing `identity:get_user(user_id=50861fe4d3f142ecbbc04ce92024a0fa)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:00.000037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd85dc0-cb66-4c1e-8aac-353c9c8554c2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:00.001677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 560/1114] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:24:59 2022] GET /identity/v3/users/50861fe4d3f142ecbbc04ce92024a0fa => generated 287 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:00.007991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aba50-71a2-4708-9578-962370f495f8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:00.113599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aba50-71a2-4708-9578-962370f495f8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:00.148406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5e3e2-acb8-4ad9-8ca3-569b82a52818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:00.149006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5e3e2-acb8-4ad9-8ca3-569b82a52818 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:00.149286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5e3e2-acb8-4ad9-8ca3-569b82a52818 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:00.149558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5e3e2-acb8-4ad9-8ca3-569b82a52818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:00.193770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:00.219762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5e3e2-acb8-4ad9-8ca3-569b82a52818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:00.221126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5e3e2-acb8-4ad9-8ca3-569b82a52818 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:00.318413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 555/1115] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:25:00 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:25:00.324421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85dc0-cb66-4c1e-8aac-353c9c8554c2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:00.368805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:00.426483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85dc0-cb66-4c1e-8aac-353c9c8554c2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:00.459660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c72e2-8ae2-4f95-a284-a3f797dd309d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:00.460270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c72e2-8ae2-4f95-a284-a3f797dd309d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:00.460544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c72e2-8ae2-4f95-a284-a3f797dd309d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:00.460809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c72e2-8ae2-4f95-a284-a3f797dd309d None admin] PATH_INFO: `/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:00.461930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c72e2-8ae2-4f95-a284-a3f797dd309d None admin] RBAC: Authorizing `identity:get_project(project_id=6f040600c7e14e0a8f0ecc791c9266fe)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:00.463343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c72e2-8ae2-4f95-a284-a3f797dd309d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:00.464953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 561/1116] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:25:00 2022] GET /identity/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:00.471574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5e3e2-acb8-4ad9-8ca3-569b82a52818 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:00.543366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:00.568229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5e3e2-acb8-4ad9-8ca3-569b82a52818 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:00.601350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c39d9-4b5b-47dc-813b-bb7b39b9de02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:00.601975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c39d9-4b5b-47dc-813b-bb7b39b9de02 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:00.602252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c39d9-4b5b-47dc-813b-bb7b39b9de02 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:00.602522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c39d9-4b5b-47dc-813b-bb7b39b9de02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:00.603248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c39d9-4b5b-47dc-813b-bb7b39b9de02 None admin] RBAC: Adding query filter params (role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=6f040600c7e14e0a8f0ecc791c9266fe, user.id=50861fe4d3f142ecbbc04ce92024a0fa) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:00.603551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c39d9-4b5b-47dc-813b-bb7b39b9de02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:00.604968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c39d9-4b5b-47dc-813b-bb7b39b9de02 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:00.610575 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 556/1117] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:25:00 2022] GET /identity/v3/role_assignments?user.id=50861fe4d3f142ecbbc04ce92024a0fa&scope.project.id=6f040600c7e14e0a8f0ecc791c9266fe&role.id=5dc7056ddc154b81939a17474a64f546 => generated 260 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:02.483512 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd75d90-f742-4caf-a34b-8703a1f93cdc None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:02.484063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd75d90-f742-4caf-a34b-8703a1f93cdc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:02.484063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd75d90-f742-4caf-a34b-8703a1f93cdc None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:02.485362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 562/1118] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:02.491264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc659c8c-1780-4fb2-aa02-dc166ef75cb7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:02.491743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc659c8c-1780-4fb2-aa02-dc166ef75cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:02.491842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc659c8c-1780-4fb2-aa02-dc166ef75cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:02.510847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dc659c8c-1780-4fb2-aa02-dc166ef75cb7 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:02.512047 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dc659c8c-1780-4fb2-aa02-dc166ef75cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:02.581136 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 557/1119] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:02 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:02.850584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd75d90-f742-4caf-a34b-8703a1f93cdc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:02.960632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd75d90-f742-4caf-a34b-8703a1f93cdc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:02.971699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:02.995695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98d017-1813-4ae5-a1e7-c25115f6d2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:02.996328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98d017-1813-4ae5-a1e7-c25115f6d2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:02.996609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98d017-1813-4ae5-a1e7-c25115f6d2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:02.996887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98d017-1813-4ae5-a1e7-c25115f6d2f1 None admin] PATH_INFO: `/v3/roles/5dc7056ddc154b81939a17474a64f546` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:02.998309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98d017-1813-4ae5-a1e7-c25115f6d2f1 None admin] RBAC: Authorizing `identity:get_role(role_id=5dc7056ddc154b81939a17474a64f546)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:02.999667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98d017-1813-4ae5-a1e7-c25115f6d2f1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:03.001004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 563/1120] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:02 2022] GET /identity/v3/roles/5dc7056ddc154b81939a17474a64f546 => generated 224 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:03.008171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc659c8c-1780-4fb2-aa02-dc166ef75cb7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:03.074611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:03.108544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc659c8c-1780-4fb2-aa02-dc166ef75cb7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:03.145596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d97ca-3f9a-4ad8-a109-d381f1ee4be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:03.146287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d97ca-3f9a-4ad8-a109-d381f1ee4be1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:03.146569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d97ca-3f9a-4ad8-a109-d381f1ee4be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:03.146839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d97ca-3f9a-4ad8-a109-d381f1ee4be1 None admin] PATH_INFO: `/v3/users/50861fe4d3f142ecbbc04ce92024a0fa` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:03.148166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d97ca-3f9a-4ad8-a109-d381f1ee4be1 None admin] RBAC: Authorizing `identity:get_user(user_id=50861fe4d3f142ecbbc04ce92024a0fa)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:03.150261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d97ca-3f9a-4ad8-a109-d381f1ee4be1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:03.151966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 558/1121] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:03 2022] GET /identity/v3/users/50861fe4d3f142ecbbc04ce92024a0fa => generated 287 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:03.158174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98d017-1813-4ae5-a1e7-c25115f6d2f1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:03.263198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98d017-1813-4ae5-a1e7-c25115f6d2f1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:03.298229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a27d0-b82f-4c6e-84b2-11d98ea96a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:03.298863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a27d0-b82f-4c6e-84b2-11d98ea96a00 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:03.299143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a27d0-b82f-4c6e-84b2-11d98ea96a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:03.299415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a27d0-b82f-4c6e-84b2-11d98ea96a00 None admin] PATH_INFO: `/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:03.300563 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a27d0-b82f-4c6e-84b2-11d98ea96a00 None admin] RBAC: Authorizing `identity:get_project(project_id=6f040600c7e14e0a8f0ecc791c9266fe)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:03.302114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a27d0-b82f-4c6e-84b2-11d98ea96a00 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:03.303767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 564/1122] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:25:03 2022] GET /identity/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:25:03.310339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d97ca-3f9a-4ad8-a109-d381f1ee4be1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:03.413698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d97ca-3f9a-4ad8-a109-d381f1ee4be1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:03.448374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af76b3-ece8-4434-a6e6-732fb64083ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:03.449057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af76b3-ece8-4434-a6e6-732fb64083ba None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:03.449334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af76b3-ece8-4434-a6e6-732fb64083ba None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:03.449627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af76b3-ece8-4434-a6e6-732fb64083ba None admin] PATH_INFO: `/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe/users/50861fe4d3f142ecbbc04ce92024a0fa/roles/5dc7056ddc154b81939a17474a64f546` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:03.451733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af76b3-ece8-4434-a6e6-732fb64083ba None admin] RBAC: Authorizing `identity:create_grant(project_id=6f040600c7e14e0a8f0ecc791c9266fe, user_id=50861fe4d3f142ecbbc04ce92024a0fa, role_id=5dc7056ddc154b81939a17474a64f546)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:03.453220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af76b3-ece8-4434-a6e6-732fb64083ba None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:03.456370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:03.469670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 559/1123] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:25:03 2022] PUT /identity/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe/users/50861fe4d3f142ecbbc04ce92024a0fa/roles/5dc7056ddc154b81939a17474a64f546 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:25:05.411274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dd559-50b2-40d2-98cf-025358d1a6c3 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:05.412328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dd559-50b2-40d2-98cf-025358d1a6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:05.412328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dd559-50b2-40d2-98cf-025358d1a6c3 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:05.413704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 565/1124] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:05.419152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dad9cb-867b-4c23-a69e-5c72151034bb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:05.419576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dad9cb-867b-4c23-a69e-5c72151034bb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:05.419705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dad9cb-867b-4c23-a69e-5c72151034bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:05.438060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e2dad9cb-867b-4c23-a69e-5c72151034bb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:05.439257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e2dad9cb-867b-4c23-a69e-5c72151034bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:05.511581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 560/1125] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:05 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:05.784067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dd559-50b2-40d2-98cf-025358d1a6c3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:05.831174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:05.887848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dd559-50b2-40d2-98cf-025358d1a6c3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:05.920522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145abcee-3d6c-4968-b6b7-cee4471cda98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:05.921146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145abcee-3d6c-4968-b6b7-cee4471cda98 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:05.921423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145abcee-3d6c-4968-b6b7-cee4471cda98 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:05.921717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145abcee-3d6c-4968-b6b7-cee4471cda98 None admin] PATH_INFO: `/v3/roles/5dc7056ddc154b81939a17474a64f546` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:05.923097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145abcee-3d6c-4968-b6b7-cee4471cda98 None admin] RBAC: Authorizing `identity:get_role(role_id=5dc7056ddc154b81939a17474a64f546)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:05.924451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145abcee-3d6c-4968-b6b7-cee4471cda98 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:05.925802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 566/1126] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:05 2022] GET /identity/v3/roles/5dc7056ddc154b81939a17474a64f546 => generated 224 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:05.932743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dad9cb-867b-4c23-a69e-5c72151034bb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:06.028512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dad9cb-867b-4c23-a69e-5c72151034bb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:06.054190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:06.063045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ed46a-85d0-44b2-a657-b9832f2d8f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:06.063673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ed46a-85d0-44b2-a657-b9832f2d8f0b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:06.063954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ed46a-85d0-44b2-a657-b9832f2d8f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:06.064223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ed46a-85d0-44b2-a657-b9832f2d8f0b None admin] PATH_INFO: `/v3/users/50861fe4d3f142ecbbc04ce92024a0fa` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:06.065354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ed46a-85d0-44b2-a657-b9832f2d8f0b None admin] RBAC: Authorizing `identity:get_user(user_id=50861fe4d3f142ecbbc04ce92024a0fa)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:06.066834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ed46a-85d0-44b2-a657-b9832f2d8f0b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:06.069010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 561/1127] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:05 2022] GET /identity/v3/users/50861fe4d3f142ecbbc04ce92024a0fa => generated 287 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:06.074894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145abcee-3d6c-4968-b6b7-cee4471cda98 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:06.170169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145abcee-3d6c-4968-b6b7-cee4471cda98 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:06.185015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:06.205016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8df024-6228-4676-9d63-36b2f7b686c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:06.205632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8df024-6228-4676-9d63-36b2f7b686c6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:06.205922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8df024-6228-4676-9d63-36b2f7b686c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:06.206174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8df024-6228-4676-9d63-36b2f7b686c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:06.279087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8df024-6228-4676-9d63-36b2f7b686c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:06.280523 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8df024-6228-4676-9d63-36b2f7b686c6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:06.381209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 567/1128] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:25:06 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:25:06.387128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ed46a-85d0-44b2-a657-b9832f2d8f0b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:06.484960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ed46a-85d0-44b2-a657-b9832f2d8f0b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:06.519499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3088a-788c-4693-aa64-5de38966b5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:06.520121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3088a-788c-4693-aa64-5de38966b5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:06.520398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3088a-788c-4693-aa64-5de38966b5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:06.520698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3088a-788c-4693-aa64-5de38966b5b7 None admin] PATH_INFO: `/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:06.521832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3088a-788c-4693-aa64-5de38966b5b7 None admin] RBAC: Authorizing `identity:get_project(project_id=6f040600c7e14e0a8f0ecc791c9266fe)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:06.523249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3088a-788c-4693-aa64-5de38966b5b7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:06.524954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 562/1129] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:25:06 2022] GET /identity/v3/projects/6f040600c7e14e0a8f0ecc791c9266fe => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:06.531361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8df024-6228-4676-9d63-36b2f7b686c6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:06.546952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:06.636774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8df024-6228-4676-9d63-36b2f7b686c6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:06.672804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20378a-cd2d-41c7-aed3-3243b7270bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:06.673406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20378a-cd2d-41c7-aed3-3243b7270bce None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:06.673715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20378a-cd2d-41c7-aed3-3243b7270bce None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:06.673985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20378a-cd2d-41c7-aed3-3243b7270bce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:06.674701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20378a-cd2d-41c7-aed3-3243b7270bce None admin] RBAC: Adding query filter params (role.id=5dc7056ddc154b81939a17474a64f546, scope.project.id=6f040600c7e14e0a8f0ecc791c9266fe, user.id=50861fe4d3f142ecbbc04ce92024a0fa) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:06.675004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20378a-cd2d-41c7-aed3-3243b7270bce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:06.676407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20378a-cd2d-41c7-aed3-3243b7270bce None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:06.682583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 568/1130] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:25:06 2022] GET /identity/v3/role_assignments?user.id=50861fe4d3f142ecbbc04ce92024a0fa&scope.project.id=6f040600c7e14e0a8f0ecc791c9266fe&role.id=5dc7056ddc154b81939a17474a64f546 => generated 610 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:08.631432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dedee-48fe-4440-9251-2cde85f1bc54 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:08.631966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dedee-48fe-4440-9251-2cde85f1bc54 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:08.631966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dedee-48fe-4440-9251-2cde85f1bc54 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:08.633238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 563/1131] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:08.638335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387cbd9-a7d0-4366-86eb-75f2f89cb708 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:08.638772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387cbd9-a7d0-4366-86eb-75f2f89cb708 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:08.638879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387cbd9-a7d0-4366-86eb-75f2f89cb708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:08.657856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f387cbd9-a7d0-4366-86eb-75f2f89cb708 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:08.659056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f387cbd9-a7d0-4366-86eb-75f2f89cb708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:08.725591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 569/1132] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:08 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:08.994751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dedee-48fe-4440-9251-2cde85f1bc54 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:09.056821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:09.095706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dedee-48fe-4440-9251-2cde85f1bc54 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:09.128424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9387e-3e44-4313-9c4e-ec596713ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:09.129042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9387e-3e44-4313-9c4e-ec596713ed71 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:09.129321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9387e-3e44-4313-9c4e-ec596713ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:09.129611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9387e-3e44-4313-9c4e-ec596713ed71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:09.131282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9387e-3e44-4313-9c4e-ec596713ed71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:09.132705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9387e-3e44-4313-9c4e-ec596713ed71 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:09.134307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 564/1133] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:25:08 2022] GET /identity/v3/domains/default => generated 200 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:09.140213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387cbd9-a7d0-4366-86eb-75f2f89cb708 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:09.150169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:09.249768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387cbd9-a7d0-4366-86eb-75f2f89cb708 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:09.282274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258da169-41a2-4723-b065-de8d73349c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:09.282877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258da169-41a2-4723-b065-de8d73349c68 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:09.283158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258da169-41a2-4723-b065-de8d73349c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:09.283437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258da169-41a2-4723-b065-de8d73349c68 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:09.284072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258da169-41a2-4723-b065-de8d73349c68 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:09.285691 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258da169-41a2-4723-b065-de8d73349c68 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:09.312407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 570/1134] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:25:09 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:25:11.279758 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16dd93-84c8-4df8-8f55-60d20f14c1d4 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:11.280397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16dd93-84c8-4df8-8f55-60d20f14c1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:11.280397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16dd93-84c8-4df8-8f55-60d20f14c1d4 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:11.281637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 565/1135] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:11.287482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4114999-1963-4353-9f0c-02bbaed2fd7c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:11.287910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4114999-1963-4353-9f0c-02bbaed2fd7c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:11.288008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4114999-1963-4353-9f0c-02bbaed2fd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:11.305798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f4114999-1963-4353-9f0c-02bbaed2fd7c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:11.306967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f4114999-1963-4353-9f0c-02bbaed2fd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:11.361901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:11.374911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 571/1136] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:11 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:11.642106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16dd93-84c8-4df8-8f55-60d20f14c1d4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:11.742597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16dd93-84c8-4df8-8f55-60d20f14c1d4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:11.770233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:11.777073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb059-ea00-499b-9160-607facceb621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:11.777716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb059-ea00-499b-9160-607facceb621 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:11.777992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb059-ea00-499b-9160-607facceb621 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:11.778259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb059-ea00-499b-9160-607facceb621 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:11.779731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eb059-ea00-499b-9160-607facceb621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:11.781010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eb059-ea00-499b-9160-607facceb621 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:11.782754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 566/1137] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:25:11 2022] GET /identity/v3/domains/default => generated 200 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:11.788482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4114999-1963-4353-9f0c-02bbaed2fd7c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:11.885830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4114999-1963-4353-9f0c-02bbaed2fd7c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:11.918820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d5428-86ac-4c11-8bee-507754abf365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:11.919340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d5428-86ac-4c11-8bee-507754abf365 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:11.919615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d5428-86ac-4c11-8bee-507754abf365 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:11.919884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d5428-86ac-4c11-8bee-507754abf365 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:11.920514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d5428-86ac-4c11-8bee-507754abf365 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:11.921947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d5428-86ac-4c11-8bee-507754abf365 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:11.946912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 572/1138] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:25:11 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:25:13.906930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e954a-9de0-4314-bb53-010522684fa7 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:13.907456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e954a-9de0-4314-bb53-010522684fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:13.907456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e954a-9de0-4314-bb53-010522684fa7 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:13.908743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 567/1139] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:13.914245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ab8c1-9f16-4663-9c32-6558b4924ecb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:13.914705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ab8c1-9f16-4663-9c32-6558b4924ecb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:13.914800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ab8c1-9f16-4663-9c32-6558b4924ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:13.932747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-908ab8c1-9f16-4663-9c32-6558b4924ecb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:13.933961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-908ab8c1-9f16-4663-9c32-6558b4924ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:13.976798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:14.002466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 573/1140] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:13 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:14.272201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e954a-9de0-4314-bb53-010522684fa7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:14.375855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e954a-9de0-4314-bb53-010522684fa7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:14.409667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bbe9a-fb0a-4810-87c8-d242a1cd50b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:14.410280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bbe9a-fb0a-4810-87c8-d242a1cd50b2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:14.410556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bbe9a-fb0a-4810-87c8-d242a1cd50b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:14.410827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bbe9a-fb0a-4810-87c8-d242a1cd50b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:14.420921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bbe9a-fb0a-4810-87c8-d242a1cd50b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9c8bbe9a-fb0a-4810-87c8-d242a1cd50b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:25:14.425673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:25:14.427511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:25:14.428522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 568/1141] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:25:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:25:14.434052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ab8c1-9f16-4663-9c32-6558b4924ecb None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:14.532318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ab8c1-9f16-4663-9c32-6558b4924ecb None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:14.568873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4b16a-b49e-4b69-a982-f2828958f3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:14.569473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4b16a-b49e-4b69-a982-f2828958f3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:14.569781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4b16a-b49e-4b69-a982-f2828958f3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:14.570182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4b16a-b49e-4b69-a982-f2828958f3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:14.570718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4b16a-b49e-4b69-a982-f2828958f3d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:14.571021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4b16a-b49e-4b69-a982-f2828958f3d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:14.572384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4b16a-b49e-4b69-a982-f2828958f3d5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:14.580184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 574/1142] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:25:14 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:14.586275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bbe9a-fb0a-4810-87c8-d242a1cd50b2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:14.645234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:14.687219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bbe9a-fb0a-4810-87c8-d242a1cd50b2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:14.720310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6da0b-a92b-4cb3-8650-ec8140a19fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:14.720930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6da0b-a92b-4cb3-8650-ec8140a19fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:14.721206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6da0b-a92b-4cb3-8650-ec8140a19fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:14.721477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6da0b-a92b-4cb3-8650-ec8140a19fe4 None admin] PATH_INFO: `/v3/users/71862f30c16a4ed7b56c0917769eb94c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:14.735126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6da0b-a92b-4cb3-8650-ec8140a19fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=71862f30c16a4ed7b56c0917769eb94c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:14.736543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6da0b-a92b-4cb3-8650-ec8140a19fe4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:14.738227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 569/1143] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:14 2022] GET /identity/v3/users/71862f30c16a4ed7b56c0917769eb94c => generated 287 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:14.745070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4b16a-b49e-4b69-a982-f2828958f3d5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:14.784828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:14.845241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4b16a-b49e-4b69-a982-f2828958f3d5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:14.879415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3e152-0db4-4a97-81de-129d196f1a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:14.880347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3e152-0db4-4a97-81de-129d196f1a96 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:14.880529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3e152-0db4-4a97-81de-129d196f1a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:14.880720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3e152-0db4-4a97-81de-129d196f1a96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:14.945774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3e152-0db4-4a97-81de-129d196f1a96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:14.947207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3e152-0db4-4a97-81de-129d196f1a96 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:14.984192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:15.044110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 575/1144] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:25:14 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:25:15.050324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6da0b-a92b-4cb3-8650-ec8140a19fe4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:15.148257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6da0b-a92b-4cb3-8650-ec8140a19fe4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:15.164772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:15.183751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2181c-8621-4865-8a23-c2cdeeca2c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:15.184372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2181c-8621-4865-8a23-c2cdeeca2c6b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:15.184648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2181c-8621-4865-8a23-c2cdeeca2c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:15.184911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2181c-8621-4865-8a23-c2cdeeca2c6b None admin] PATH_INFO: `/v3/projects/fff0b6fe70644daba276f9b0c74af3ea` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:15.186048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2181c-8621-4865-8a23-c2cdeeca2c6b None admin] RBAC: Authorizing `identity:get_project(project_id=fff0b6fe70644daba276f9b0c74af3ea)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:15.187457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2181c-8621-4865-8a23-c2cdeeca2c6b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:15.189238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 570/1145] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:25:15 2022] GET /identity/v3/projects/fff0b6fe70644daba276f9b0c74af3ea => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:15.195506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3e152-0db4-4a97-81de-129d196f1a96 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:15.298162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3e152-0db4-4a97-81de-129d196f1a96 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:15.332000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587fd2f-7184-416b-aefb-42fc3e7bd162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:15.332589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587fd2f-7184-416b-aefb-42fc3e7bd162 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:15.332866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587fd2f-7184-416b-aefb-42fc3e7bd162 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:15.333943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587fd2f-7184-416b-aefb-42fc3e7bd162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:15.334661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587fd2f-7184-416b-aefb-42fc3e7bd162 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=fff0b6fe70644daba276f9b0c74af3ea, user.id=71862f30c16a4ed7b56c0917769eb94c) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:15.334975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587fd2f-7184-416b-aefb-42fc3e7bd162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:15.336388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587fd2f-7184-416b-aefb-42fc3e7bd162 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:15.341695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 576/1146] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:25:15 2022] GET /identity/v3/role_assignments?user.id=71862f30c16a4ed7b56c0917769eb94c&scope.project.id=fff0b6fe70644daba276f9b0c74af3ea&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 260 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:17.216306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023fdb6-4176-4397-a477-0c8dee5aa822 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:17.216869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023fdb6-4176-4397-a477-0c8dee5aa822 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:17.216869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023fdb6-4176-4397-a477-0c8dee5aa822 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:17.218238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 571/1147] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:17.223389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ae60d-dfb4-4a61-ac3d-1d95ca741c77 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:17.223834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ae60d-dfb4-4a61-ac3d-1d95ca741c77 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:17.224415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ae60d-dfb4-4a61-ac3d-1d95ca741c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:17.230649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:17.244456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-132ae60d-dfb4-4a61-ac3d-1d95ca741c77 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:17.245652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-132ae60d-dfb4-4a61-ac3d-1d95ca741c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:17.311561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 577/1148] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:17 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:17.580816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023fdb6-4176-4397-a477-0c8dee5aa822 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:17.683867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023fdb6-4176-4397-a477-0c8dee5aa822 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:17.716032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdad835-9489-4acc-a021-f660024ba465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:17.716647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdad835-9489-4acc-a021-f660024ba465 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:17.716920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdad835-9489-4acc-a021-f660024ba465 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:17.717188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdad835-9489-4acc-a021-f660024ba465 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:17.728766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdad835-9489-4acc-a021-f660024ba465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-6bdad835-9489-4acc-a021-f660024ba465 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:25:17.733469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:25:17.734808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:25:17.735658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 572/1149] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:25:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:25:17.740874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ae60d-dfb4-4a61-ac3d-1d95ca741c77 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:17.838113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ae60d-dfb4-4a61-ac3d-1d95ca741c77 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:17.867834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:17.873216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be232f-8212-4ff6-a635-b22621827a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:17.873838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be232f-8212-4ff6-a635-b22621827a1c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:17.874114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be232f-8212-4ff6-a635-b22621827a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:17.874383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be232f-8212-4ff6-a635-b22621827a1c None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:17.875036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be232f-8212-4ff6-a635-b22621827a1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:17.875339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be232f-8212-4ff6-a635-b22621827a1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:17.876687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be232f-8212-4ff6-a635-b22621827a1c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:17.884735 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 578/1150] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:25:17 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:17.890728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdad835-9489-4acc-a021-f660024ba465 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:17.936724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:17.990008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdad835-9489-4acc-a021-f660024ba465 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:18.023372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ae38a-94e9-4138-9505-011beb246d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:18.023986 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ae38a-94e9-4138-9505-011beb246d55 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:18.024260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ae38a-94e9-4138-9505-011beb246d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:18.024533 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ae38a-94e9-4138-9505-011beb246d55 None admin] PATH_INFO: `/v3/users/71862f30c16a4ed7b56c0917769eb94c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:18.025680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ae38a-94e9-4138-9505-011beb246d55 None admin] RBAC: Authorizing `identity:get_user(user_id=71862f30c16a4ed7b56c0917769eb94c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:18.027112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ae38a-94e9-4138-9505-011beb246d55 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:18.028683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 573/1151] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:17 2022] GET /identity/v3/users/71862f30c16a4ed7b56c0917769eb94c => generated 287 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:18.034812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be232f-8212-4ff6-a635-b22621827a1c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:18.132592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be232f-8212-4ff6-a635-b22621827a1c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:18.167283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffa681-e68b-4146-9e3f-4b490e8d3537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:18.167899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffa681-e68b-4146-9e3f-4b490e8d3537 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:18.168176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffa681-e68b-4146-9e3f-4b490e8d3537 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:18.168444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffa681-e68b-4146-9e3f-4b490e8d3537 None admin] PATH_INFO: `/v3/projects/fff0b6fe70644daba276f9b0c74af3ea` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:18.169531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffa681-e68b-4146-9e3f-4b490e8d3537 None admin] RBAC: Authorizing `identity:get_project(project_id=fff0b6fe70644daba276f9b0c74af3ea)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:18.170969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffa681-e68b-4146-9e3f-4b490e8d3537 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:18.172575 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 579/1152] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:25:18 2022] GET /identity/v3/projects/fff0b6fe70644daba276f9b0c74af3ea => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:18.178816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ae38a-94e9-4138-9505-011beb246d55 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:18.278125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ae38a-94e9-4138-9505-011beb246d55 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:18.291244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:18.312892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8796336-3ffa-47b0-8a8b-30466f0736e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:18.313536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8796336-3ffa-47b0-8a8b-30466f0736e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:18.313847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8796336-3ffa-47b0-8a8b-30466f0736e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:18.314114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8796336-3ffa-47b0-8a8b-30466f0736e3 None admin] PATH_INFO: `/v3/projects/fff0b6fe70644daba276f9b0c74af3ea/users/71862f30c16a4ed7b56c0917769eb94c/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:18.315960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8796336-3ffa-47b0-8a8b-30466f0736e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=fff0b6fe70644daba276f9b0c74af3ea, user_id=71862f30c16a4ed7b56c0917769eb94c, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:18.318085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8796336-3ffa-47b0-8a8b-30466f0736e3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:18.332210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 574/1153] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:25:18 2022] PUT /identity/v3/projects/fff0b6fe70644daba276f9b0c74af3ea/users/71862f30c16a4ed7b56c0917769eb94c/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:25:20.244298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de2a9-75b5-421a-8c69-93abc3cdf524 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:20.244852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de2a9-75b5-421a-8c69-93abc3cdf524 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:20.244852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de2a9-75b5-421a-8c69-93abc3cdf524 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:20.246196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 580/1154] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:20.251592 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e0995-c402-48ad-8ce2-12334cbec7e0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:20.252003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e0995-c402-48ad-8ce2-12334cbec7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:20.252135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e0995-c402-48ad-8ce2-12334cbec7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:20.270055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-db9e0995-c402-48ad-8ce2-12334cbec7e0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:20.271738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-db9e0995-c402-48ad-8ce2-12334cbec7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:20.337676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 575/1155] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:20 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:25:20.605079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de2a9-75b5-421a-8c69-93abc3cdf524 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:20.668059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:20.707763 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de2a9-75b5-421a-8c69-93abc3cdf524 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:20.739971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0af1be-5972-463c-938d-fc1949efd31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:20.740577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af1be-5972-463c-938d-fc1949efd31c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:20.740851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af1be-5972-463c-938d-fc1949efd31c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:20.741115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af1be-5972-463c-938d-fc1949efd31c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:20.751192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0af1be-5972-463c-938d-fc1949efd31c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6d0af1be-5972-463c-938d-fc1949efd31c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:25:20.756565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:25:20.757706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:25:20.758593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 581/1156] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:25:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:25:20.763747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e0995-c402-48ad-8ce2-12334cbec7e0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:20.860543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e0995-c402-48ad-8ce2-12334cbec7e0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:20.861973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:20.896678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f6eb7-591c-470c-b4e5-d92ab0f656d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:20.897278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f6eb7-591c-470c-b4e5-d92ab0f656d6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:20.897560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f6eb7-591c-470c-b4e5-d92ab0f656d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:20.897862 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f6eb7-591c-470c-b4e5-d92ab0f656d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:20.898519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f6eb7-591c-470c-b4e5-d92ab0f656d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:20.899785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f6eb7-591c-470c-b4e5-d92ab0f656d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:20.901147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f6eb7-591c-470c-b4e5-d92ab0f656d6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:20.909324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 576/1157] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:25:20 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:20.915294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0af1be-5972-463c-938d-fc1949efd31c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:21.018128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0af1be-5972-463c-938d-fc1949efd31c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:21.040076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:21.052242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52f260-e8c2-49a2-ae49-9f8ab712c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:21.052943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52f260-e8c2-49a2-ae49-9f8ab712c14d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:21.053227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52f260-e8c2-49a2-ae49-9f8ab712c14d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:21.053498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52f260-e8c2-49a2-ae49-9f8ab712c14d None admin] PATH_INFO: `/v3/users/71862f30c16a4ed7b56c0917769eb94c` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:21.054639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52f260-e8c2-49a2-ae49-9f8ab712c14d None admin] RBAC: Authorizing `identity:get_user(user_id=71862f30c16a4ed7b56c0917769eb94c)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:21.056056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52f260-e8c2-49a2-ae49-9f8ab712c14d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:21.057654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 582/1158] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:20 2022] GET /identity/v3/users/71862f30c16a4ed7b56c0917769eb94c => generated 287 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:21.063578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f6eb7-591c-470c-b4e5-d92ab0f656d6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:21.162344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f6eb7-591c-470c-b4e5-d92ab0f656d6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:21.195771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad7261-4fb1-4b4a-aed6-911d3f39a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:21.196366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad7261-4fb1-4b4a-aed6-911d3f39a087 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:21.196649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad7261-4fb1-4b4a-aed6-911d3f39a087 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:21.196919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad7261-4fb1-4b4a-aed6-911d3f39a087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:21.228293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:21.264734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad7261-4fb1-4b4a-aed6-911d3f39a087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:21.266214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad7261-4fb1-4b4a-aed6-911d3f39a087 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:21.362803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 577/1159] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:25:21 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:25:21.369498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52f260-e8c2-49a2-ae49-9f8ab712c14d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:21.468894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52f260-e8c2-49a2-ae49-9f8ab712c14d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:21.501950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5f03f-858a-4a11-ab1a-72328aaf108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:21.502561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5f03f-858a-4a11-ab1a-72328aaf108e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:21.502875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5f03f-858a-4a11-ab1a-72328aaf108e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:21.503103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5f03f-858a-4a11-ab1a-72328aaf108e None admin] PATH_INFO: `/v3/projects/fff0b6fe70644daba276f9b0c74af3ea` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:21.504192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5f03f-858a-4a11-ab1a-72328aaf108e None admin] RBAC: Authorizing `identity:get_project(project_id=fff0b6fe70644daba276f9b0c74af3ea)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:21.506071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5f03f-858a-4a11-ab1a-72328aaf108e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:21.507674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 583/1160] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:25:21 2022] GET /identity/v3/projects/fff0b6fe70644daba276f9b0c74af3ea => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:21.513802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad7261-4fb1-4b4a-aed6-911d3f39a087 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:21.574193 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:21.620475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad7261-4fb1-4b4a-aed6-911d3f39a087 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:21.654686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f43b81-acef-4831-99e8-fcc398dd6275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:21.655469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f43b81-acef-4831-99e8-fcc398dd6275 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:21.655469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f43b81-acef-4831-99e8-fcc398dd6275 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:21.655722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f43b81-acef-4831-99e8-fcc398dd6275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:21.656444 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f43b81-acef-4831-99e8-fcc398dd6275 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=fff0b6fe70644daba276f9b0c74af3ea, user.id=71862f30c16a4ed7b56c0917769eb94c) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:21.656771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f43b81-acef-4831-99e8-fcc398dd6275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:21.658247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f43b81-acef-4831-99e8-fcc398dd6275 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:21.664726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 578/1161] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:25:21 2022] GET /identity/v3/role_assignments?user.id=71862f30c16a4ed7b56c0917769eb94c&scope.project.id=fff0b6fe70644daba276f9b0c74af3ea&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 610 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:23.640760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4b55c-4835-42c9-b699-cceb2994ac60 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:23.641338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4b55c-4835-42c9-b699-cceb2994ac60 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:23.641338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4b55c-4835-42c9-b699-cceb2994ac60 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:23.642740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 584/1162] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:23.649041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddd754-1877-4f61-873e-a6335ce3ae4f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:23.649440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddd754-1877-4f61-873e-a6335ce3ae4f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:23.649630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddd754-1877-4f61-873e-a6335ce3ae4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:23.668121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9cddd754-1877-4f61-873e-a6335ce3ae4f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:23.669301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9cddd754-1877-4f61-873e-a6335ce3ae4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:23.734977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 579/1163] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:23 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:24.006793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4b55c-4835-42c9-b699-cceb2994ac60 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:24.064810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:24.107240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4b55c-4835-42c9-b699-cceb2994ac60 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:24.140427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cfcf4-7cb1-491b-a051-d843fcce7f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:24.141012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cfcf4-7cb1-491b-a051-d843fcce7f29 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:24.141287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cfcf4-7cb1-491b-a051-d843fcce7f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:24.141552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cfcf4-7cb1-491b-a051-d843fcce7f29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:24.207508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cfcf4-7cb1-491b-a051-d843fcce7f29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:24.209258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cfcf4-7cb1-491b-a051-d843fcce7f29 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:24.263152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:24.306955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 585/1164] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:25:24 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:25:24.312923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddd754-1877-4f61-873e-a6335ce3ae4f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:24.369143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:24.408929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddd754-1877-4f61-873e-a6335ce3ae4f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:24.442030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f84ac-1b28-4526-8652-2f2e99fcf216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:24.442807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f84ac-1b28-4526-8652-2f2e99fcf216 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:24.443088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f84ac-1b28-4526-8652-2f2e99fcf216 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:24.443362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f84ac-1b28-4526-8652-2f2e99fcf216 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:24.449929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f84ac-1b28-4526-8652-2f2e99fcf216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:24.451387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f84ac-1b28-4526-8652-2f2e99fcf216 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-080f84ac-1b28-4526-8652-2f2e99fcf216 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:25:24.462464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:25:24.463674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:25:24.464562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 580/1165] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:25:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:25:24.469597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cfcf4-7cb1-491b-a051-d843fcce7f29 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:24.569463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cfcf4-7cb1-491b-a051-d843fcce7f29 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:24.602433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e2e42-e5f2-4c8c-b3c0-ca7a69bf1447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:24.603025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2e42-e5f2-4c8c-b3c0-ca7a69bf1447 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:24.603304 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2e42-e5f2-4c8c-b3c0-ca7a69bf1447 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:24.603573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2e42-e5f2-4c8c-b3c0-ca7a69bf1447 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:24.604215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2e42-e5f2-4c8c-b3c0-ca7a69bf1447 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:24.604510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2e42-e5f2-4c8c-b3c0-ca7a69bf1447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:24.605875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2e42-e5f2-4c8c-b3c0-ca7a69bf1447 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:24.613036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 586/1166] 10.222.0.100 () {56 vars in 1120 bytes} [Thu Sep 1 17:25:24 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:26.517488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd191cb-4390-441c-91f4-25d42f90abf5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:26.518044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd191cb-4390-441c-91f4-25d42f90abf5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:26.518044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd191cb-4390-441c-91f4-25d42f90abf5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:26.519410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 581/1167] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:26.525201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccf25a-e0d0-425b-bad2-e21f3798994d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:26.525653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccf25a-e0d0-425b-bad2-e21f3798994d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:26.525780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccf25a-e0d0-425b-bad2-e21f3798994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:26.543793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-eaccf25a-e0d0-425b-bad2-e21f3798994d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:26.544974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-eaccf25a-e0d0-425b-bad2-e21f3798994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:26.560964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:26.611572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 587/1168] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:26 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:26.879238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd191cb-4390-441c-91f4-25d42f90abf5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:26.980379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd191cb-4390-441c-91f4-25d42f90abf5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:26.996733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:27.013385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d4a42-f165-4d87-aec5-ec17b96ef24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:27.014003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d4a42-f165-4d87-aec5-ec17b96ef24b None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:27.014278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d4a42-f165-4d87-aec5-ec17b96ef24b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:27.014542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d4a42-f165-4d87-aec5-ec17b96ef24b None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:27.015189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d4a42-f165-4d87-aec5-ec17b96ef24b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:27.016533 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d4a42-f165-4d87-aec5-ec17b96ef24b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:27.032018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 582/1169] 10.222.0.100 () {60 vars in 1143 bytes} [Thu Sep 1 17:25:26 2022] POST /identity/v3/domains => generated 268 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:25:28.979701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3c2a6-9ef5-47fd-847a-81a461a2ff30 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:28.980227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3c2a6-9ef5-47fd-847a-81a461a2ff30 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:28.980227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3c2a6-9ef5-47fd-847a-81a461a2ff30 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:28.981487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 588/1170] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:28.986644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5ac5d-f094-42e1-bec0-fc65dbf60a06 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:28.987031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5ac5d-f094-42e1-bec0-fc65dbf60a06 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:28.987212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5ac5d-f094-42e1-bec0-fc65dbf60a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:29.005613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e4b5ac5d-f094-42e1-bec0-fc65dbf60a06 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:29.006797 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e4b5ac5d-f094-42e1-bec0-fc65dbf60a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:29.071264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 583/1171] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:28 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:29.345104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3c2a6-9ef5-47fd-847a-81a461a2ff30 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:29.444413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3c2a6-9ef5-47fd-847a-81a461a2ff30 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:29.479155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0737377-5290-43fd-aa0e-260e5aaa9c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:29.479815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0737377-5290-43fd-aa0e-260e5aaa9c78 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:29.480103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0737377-5290-43fd-aa0e-260e5aaa9c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:29.480368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0737377-5290-43fd-aa0e-260e5aaa9c78 None admin] PATH_INFO: `/v3/domains/93985be101e543c1875902b394894d4e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:29.490609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0737377-5290-43fd-aa0e-260e5aaa9c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=93985be101e543c1875902b394894d4e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:29.492029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0737377-5290-43fd-aa0e-260e5aaa9c78 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:29.493619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 589/1172] 10.222.0.100 () {56 vars in 1188 bytes} [Thu Sep 1 17:25:29 2022] GET /identity/v3/domains/93985be101e543c1875902b394894d4e => generated 268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:29.499654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5ac5d-f094-42e1-bec0-fc65dbf60a06 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:29.597652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5ac5d-f094-42e1-bec0-fc65dbf60a06 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:29.607163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:29.632676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89628ce9-5a75-4231-8059-ba6c9813a86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:29.633275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89628ce9-5a75-4231-8059-ba6c9813a86c None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:29.633552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89628ce9-5a75-4231-8059-ba6c9813a86c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:29.633869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89628ce9-5a75-4231-8059-ba6c9813a86c None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:29.634518 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89628ce9-5a75-4231-8059-ba6c9813a86c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:29.635940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89628ce9-5a75-4231-8059-ba6c9813a86c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:29.667473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 584/1173] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:25:29 2022] POST /identity/v3/projects => generated 362 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:25:31.590504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d8719-b7d7-4f77-b425-9145696ed5d7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:31.591065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d8719-b7d7-4f77-b425-9145696ed5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:31.591065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d8719-b7d7-4f77-b425-9145696ed5d7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:31.592372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 590/1174] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:31.597529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8296469-3832-427c-a6a4-882271bfe496 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:31.598740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8296469-3832-427c-a6a4-882271bfe496 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:31.598740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8296469-3832-427c-a6a4-882271bfe496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:31.616687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f8296469-3832-427c-a6a4-882271bfe496 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:31.617945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f8296469-3832-427c-a6a4-882271bfe496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:31.683438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 585/1175] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:31 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:31.952164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d8719-b7d7-4f77-b425-9145696ed5d7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:31.967146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:32.053454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d8719-b7d7-4f77-b425-9145696ed5d7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:32.085500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c951a25-a9ea-4e74-a704-3acc108104b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:32.086140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c951a25-a9ea-4e74-a704-3acc108104b3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:32.086418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c951a25-a9ea-4e74-a704-3acc108104b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:32.086686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c951a25-a9ea-4e74-a704-3acc108104b3 None admin] PATH_INFO: `/v3/domains/93985be101e543c1875902b394894d4e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:32.087788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c951a25-a9ea-4e74-a704-3acc108104b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=93985be101e543c1875902b394894d4e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:32.089204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c951a25-a9ea-4e74-a704-3acc108104b3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:32.090793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 591/1176] 10.222.0.100 () {56 vars in 1188 bytes} [Thu Sep 1 17:25:31 2022] GET /identity/v3/domains/93985be101e543c1875902b394894d4e => generated 268 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:32.096958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8296469-3832-427c-a6a4-882271bfe496 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:32.183004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:32.192967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8296469-3832-427c-a6a4-882271bfe496 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:32.227375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b962f-0cd9-47c8-bc2f-f7f64e70929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:32.227893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b962f-0cd9-47c8-bc2f-f7f64e70929a None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:32.228167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b962f-0cd9-47c8-bc2f-f7f64e70929a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:32.228439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b962f-0cd9-47c8-bc2f-f7f64e70929a None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:32.229077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b962f-0cd9-47c8-bc2f-f7f64e70929a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:32.230521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b962f-0cd9-47c8-bc2f-f7f64e70929a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:32.252012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 586/1177] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:25:32 2022] POST /identity/v3/users => generated 312 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:25:34.220821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a140596-cbec-4bd7-8426-9b0feb6ae290 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:34.221382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a140596-cbec-4bd7-8426-9b0feb6ae290 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:34.221382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a140596-cbec-4bd7-8426-9b0feb6ae290 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:34.222731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 592/1178] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:34.228002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc44286-dc12-456a-a960-15f9e6472435 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:34.228429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc44286-dc12-456a-a960-15f9e6472435 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:34.228557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc44286-dc12-456a-a960-15f9e6472435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:34.246481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4cc44286-dc12-456a-a960-15f9e6472435 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:34.247672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4cc44286-dc12-456a-a960-15f9e6472435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:34.313815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 587/1179] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:34 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:34.588696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a140596-cbec-4bd7-8426-9b0feb6ae290 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:34.669863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:34.689333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a140596-cbec-4bd7-8426-9b0feb6ae290 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:34.721762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9a466-4d93-41ae-a27b-11b6ed4ee00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:34.722378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9a466-4d93-41ae-a27b-11b6ed4ee00f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:34.722652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9a466-4d93-41ae-a27b-11b6ed4ee00f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:34.722918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9a466-4d93-41ae-a27b-11b6ed4ee00f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:34.733983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9a466-4d93-41ae-a27b-11b6ed4ee00f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8fe9a466-4d93-41ae-a27b-11b6ed4ee00f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:25:34.738740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:25:34.739931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:25:34.741000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 593/1180] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:25:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:25:34.746492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc44286-dc12-456a-a960-15f9e6472435 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:34.822287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:34.846240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc44286-dc12-456a-a960-15f9e6472435 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:34.878986 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1276a-0d22-48fc-ad27-39e915cb5484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:34.880191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1276a-0d22-48fc-ad27-39e915cb5484 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:34.881003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1276a-0d22-48fc-ad27-39e915cb5484 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:34.881003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1276a-0d22-48fc-ad27-39e915cb5484 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:34.881683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1276a-0d22-48fc-ad27-39e915cb5484 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:34.881983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1276a-0d22-48fc-ad27-39e915cb5484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:34.883326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1276a-0d22-48fc-ad27-39e915cb5484 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:34.891065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 588/1181] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:25:34 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:34.897506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9a466-4d93-41ae-a27b-11b6ed4ee00f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:34.996423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9a466-4d93-41ae-a27b-11b6ed4ee00f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:35.030932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfe953-819c-4789-b5fc-1cd04965d1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:35.031546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfe953-819c-4789-b5fc-1cd04965d1fc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:35.031824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfe953-819c-4789-b5fc-1cd04965d1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:35.032093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfe953-819c-4789-b5fc-1cd04965d1fc None admin] PATH_INFO: `/v3/users/4fc023aee4214fcb871bea92003ba310` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:35.042972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:35.046551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfe953-819c-4789-b5fc-1cd04965d1fc None admin] RBAC: Authorizing `identity:get_user(user_id=4fc023aee4214fcb871bea92003ba310)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:35.048529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfe953-819c-4789-b5fc-1cd04965d1fc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:35.050226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 594/1182] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:34 2022] GET /identity/v3/users/4fc023aee4214fcb871bea92003ba310 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:35.056355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1276a-0d22-48fc-ad27-39e915cb5484 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:35.155103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1276a-0d22-48fc-ad27-39e915cb5484 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:35.189890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fa2ba-d890-4878-8352-36668f9e5bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:35.190486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fa2ba-d890-4878-8352-36668f9e5bde None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:35.190764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fa2ba-d890-4878-8352-36668f9e5bde None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:35.191034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fa2ba-d890-4878-8352-36668f9e5bde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:35.200710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:35.258922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fa2ba-d890-4878-8352-36668f9e5bde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:35.260361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fa2ba-d890-4878-8352-36668f9e5bde None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:35.354747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 589/1183] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:25:35 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:25:35.360898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfe953-819c-4789-b5fc-1cd04965d1fc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:35.459563 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfe953-819c-4789-b5fc-1cd04965d1fc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:35.493535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89009045-5d82-4d2b-ab8f-d6fc0ddae7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:35.494184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89009045-5d82-4d2b-ab8f-d6fc0ddae7fc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:35.494460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89009045-5d82-4d2b-ab8f-d6fc0ddae7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:35.494728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89009045-5d82-4d2b-ab8f-d6fc0ddae7fc None admin] PATH_INFO: `/v3/projects/a4199bd51bd44961a9b4ce3dd6b64f66` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:35.495818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89009045-5d82-4d2b-ab8f-d6fc0ddae7fc None admin] RBAC: Authorizing `identity:get_project(project_id=a4199bd51bd44961a9b4ce3dd6b64f66)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:35.497226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89009045-5d82-4d2b-ab8f-d6fc0ddae7fc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:35.498856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 595/1184] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:25:35 2022] GET /identity/v3/projects/a4199bd51bd44961a9b4ce3dd6b64f66 => generated 362 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:35.505749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fa2ba-d890-4878-8352-36668f9e5bde None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:35.544641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:35.602863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fa2ba-d890-4878-8352-36668f9e5bde None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:35.641096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508989df-4a2a-477b-b836-91dfa9122d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:35.641721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508989df-4a2a-477b-b836-91dfa9122d66 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:35.642000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508989df-4a2a-477b-b836-91dfa9122d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:35.642271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508989df-4a2a-477b-b836-91dfa9122d66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:35.642985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508989df-4a2a-477b-b836-91dfa9122d66 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=a4199bd51bd44961a9b4ce3dd6b64f66, user.id=4fc023aee4214fcb871bea92003ba310) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:35.643290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508989df-4a2a-477b-b836-91dfa9122d66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:35.644714 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508989df-4a2a-477b-b836-91dfa9122d66 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:35.650137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 590/1185] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:25:35 2022] GET /identity/v3/role_assignments?user.id=4fc023aee4214fcb871bea92003ba310&scope.project.id=a4199bd51bd44961a9b4ce3dd6b64f66&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 260 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:37.610488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b6409-0053-4253-aab5-23a834c05644 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:37.611028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b6409-0053-4253-aab5-23a834c05644 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:37.611028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b6409-0053-4253-aab5-23a834c05644 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:37.612343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 596/1186] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:37.618123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00832f8-6823-4110-ba34-e53504f16a29 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:37.618514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00832f8-6823-4110-ba34-e53504f16a29 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:37.618663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00832f8-6823-4110-ba34-e53504f16a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:37.636828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b00832f8-6823-4110-ba34-e53504f16a29 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:37.638035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b00832f8-6823-4110-ba34-e53504f16a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:37.705433 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 591/1187] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:37 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:37.986963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b6409-0053-4253-aab5-23a834c05644 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:38.052486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:38.088048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b6409-0053-4253-aab5-23a834c05644 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:38.121648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6fc4c-e1a4-4ea6-80c9-0713f7127a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:38.122272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6fc4c-e1a4-4ea6-80c9-0713f7127a6a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:38.122555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6fc4c-e1a4-4ea6-80c9-0713f7127a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:38.122827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6fc4c-e1a4-4ea6-80c9-0713f7127a6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:38.133219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6fc4c-e1a4-4ea6-80c9-0713f7127a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6fc6fc4c-e1a4-4ea6-80c9-0713f7127a6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:25:38.138072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:25:38.139206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:25:38.140396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 597/1188] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:25:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:25:38.145863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00832f8-6823-4110-ba34-e53504f16a29 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:38.175539 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:38.243042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00832f8-6823-4110-ba34-e53504f16a29 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:38.276051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be53773-7cfa-4910-8284-4393abdd5d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:38.276646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be53773-7cfa-4910-8284-4393abdd5d3e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:38.276924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be53773-7cfa-4910-8284-4393abdd5d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:38.277191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be53773-7cfa-4910-8284-4393abdd5d3e None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:38.277875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be53773-7cfa-4910-8284-4393abdd5d3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:38.278176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be53773-7cfa-4910-8284-4393abdd5d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:38.279784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be53773-7cfa-4910-8284-4393abdd5d3e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:38.288191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 592/1189] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:25:38 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:38.294059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6fc4c-e1a4-4ea6-80c9-0713f7127a6a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:38.391901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6fc4c-e1a4-4ea6-80c9-0713f7127a6a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:38.418978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:38.428162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d5671-29d2-489b-9565-c116a2345ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:38.428790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5671-29d2-489b-9565-c116a2345ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:38.429071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5671-29d2-489b-9565-c116a2345ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:38.429345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5671-29d2-489b-9565-c116a2345ef1 None admin] PATH_INFO: `/v3/users/4fc023aee4214fcb871bea92003ba310` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:38.430491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d5671-29d2-489b-9565-c116a2345ef1 None admin] RBAC: Authorizing `identity:get_user(user_id=4fc023aee4214fcb871bea92003ba310)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:38.431921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d5671-29d2-489b-9565-c116a2345ef1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:38.433491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 598/1190] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:38 2022] GET /identity/v3/users/4fc023aee4214fcb871bea92003ba310 => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:38.439508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be53773-7cfa-4910-8284-4393abdd5d3e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:38.532104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:38.539020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be53773-7cfa-4910-8284-4393abdd5d3e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:38.571493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05054d52-0885-40d4-be65-c8d34fa2e697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:38.572113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05054d52-0885-40d4-be65-c8d34fa2e697 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:38.572390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05054d52-0885-40d4-be65-c8d34fa2e697 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:38.572663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05054d52-0885-40d4-be65-c8d34fa2e697 None admin] PATH_INFO: `/v3/projects/a4199bd51bd44961a9b4ce3dd6b64f66` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:38.573793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05054d52-0885-40d4-be65-c8d34fa2e697 None admin] RBAC: Authorizing `identity:get_project(project_id=a4199bd51bd44961a9b4ce3dd6b64f66)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:38.575220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05054d52-0885-40d4-be65-c8d34fa2e697 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:38.576828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 593/1191] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:25:38 2022] GET /identity/v3/projects/a4199bd51bd44961a9b4ce3dd6b64f66 => generated 362 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:38.583535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d5671-29d2-489b-9565-c116a2345ef1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:38.680853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d5671-29d2-489b-9565-c116a2345ef1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:38.714727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108eb558-9498-45e5-8482-d9246b677e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:38.715379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eb558-9498-45e5-8482-d9246b677e76 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:38.715660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eb558-9498-45e5-8482-d9246b677e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:38.715938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eb558-9498-45e5-8482-d9246b677e76 None admin] PATH_INFO: `/v3/projects/a4199bd51bd44961a9b4ce3dd6b64f66/users/4fc023aee4214fcb871bea92003ba310/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:38.717844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108eb558-9498-45e5-8482-d9246b677e76 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4199bd51bd44961a9b4ce3dd6b64f66, user_id=4fc023aee4214fcb871bea92003ba310, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:38.719270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108eb558-9498-45e5-8482-d9246b677e76 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:38.731715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 599/1192] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:25:38 2022] PUT /identity/v3/projects/a4199bd51bd44961a9b4ce3dd6b64f66/users/4fc023aee4214fcb871bea92003ba310/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:25:40.665650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c786f5-5350-410c-a03e-c65afea54618 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:40.666121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c786f5-5350-410c-a03e-c65afea54618 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:40.666221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c786f5-5350-410c-a03e-c65afea54618 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:40.667547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 594/1193] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:25:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:25:40.672822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502c8d4-c401-436f-88a1-392809a9ffff None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:40.673221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502c8d4-c401-436f-88a1-392809a9ffff None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:40.673373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502c8d4-c401-436f-88a1-392809a9ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:40.691728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0502c8d4-c401-436f-88a1-392809a9ffff None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:25:40.693665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0502c8d4-c401-436f-88a1-392809a9ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:25:40.742756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:40.761275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 600/1194] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:25:40 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:25:41.033272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c786f5-5350-410c-a03e-c65afea54618 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:41.135828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c786f5-5350-410c-a03e-c65afea54618 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:41.168951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54549e0a-813b-4284-b685-7b221104d682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:41.169565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54549e0a-813b-4284-b685-7b221104d682 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:41.169871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54549e0a-813b-4284-b685-7b221104d682 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:41.170142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54549e0a-813b-4284-b685-7b221104d682 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:41.180370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54549e0a-813b-4284-b685-7b221104d682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-54549e0a-813b-4284-b685-7b221104d682 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:25:41.185126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:25:41.186244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:25:41.187115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 595/1195] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:25:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:25:41.192463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502c8d4-c401-436f-88a1-392809a9ffff None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:41.289482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502c8d4-c401-436f-88a1-392809a9ffff None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:41.321819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cdb14-cc09-4e65-b6aa-297d83ff20bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:41.322407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cdb14-cc09-4e65-b6aa-297d83ff20bb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:41.322682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cdb14-cc09-4e65-b6aa-297d83ff20bb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:41.322949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cdb14-cc09-4e65-b6aa-297d83ff20bb None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:41.323598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cdb14-cc09-4e65-b6aa-297d83ff20bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:41.323893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cdb14-cc09-4e65-b6aa-297d83ff20bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:41.325240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cdb14-cc09-4e65-b6aa-297d83ff20bb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:41.333747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 601/1196] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:25:41 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:41.339495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54549e0a-813b-4284-b685-7b221104d682 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:41.364690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:41.438578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54549e0a-813b-4284-b685-7b221104d682 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:41.472434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bd001-2c2f-4bb2-acd3-6a2e8d56107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:41.473069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bd001-2c2f-4bb2-acd3-6a2e8d56107c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:41.474023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bd001-2c2f-4bb2-acd3-6a2e8d56107c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:41.474461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bd001-2c2f-4bb2-acd3-6a2e8d56107c None admin] PATH_INFO: `/v3/users/4fc023aee4214fcb871bea92003ba310` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:41.475583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bd001-2c2f-4bb2-acd3-6a2e8d56107c None admin] RBAC: Authorizing `identity:get_user(user_id=4fc023aee4214fcb871bea92003ba310)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:41.477011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bd001-2c2f-4bb2-acd3-6a2e8d56107c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:41.478619 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 596/1197] 10.222.0.100 () {56 vars in 1182 bytes} [Thu Sep 1 17:25:41 2022] GET /identity/v3/users/4fc023aee4214fcb871bea92003ba310 => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:41.484713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cdb14-cc09-4e65-b6aa-297d83ff20bb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:41.531002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:41.584824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cdb14-cc09-4e65-b6aa-297d83ff20bb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:41.618641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38ba61-ff30-47b5-9657-02c9d591ee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:41.619232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38ba61-ff30-47b5-9657-02c9d591ee94 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:41.619529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38ba61-ff30-47b5-9657-02c9d591ee94 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:41.619796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38ba61-ff30-47b5-9657-02c9d591ee94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:41.687915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38ba61-ff30-47b5-9657-02c9d591ee94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:41.689382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38ba61-ff30-47b5-9657-02c9d591ee94 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:41.734897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:41.788644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 602/1198] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:25:41 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:25:41.794554 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bd001-2c2f-4bb2-acd3-6a2e8d56107c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:41.883229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:41.893377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bd001-2c2f-4bb2-acd3-6a2e8d56107c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:41.927983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78827b76-2373-481a-bd7a-bde4f6917261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:41.928598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78827b76-2373-481a-bd7a-bde4f6917261 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:41.928885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78827b76-2373-481a-bd7a-bde4f6917261 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:41.929156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78827b76-2373-481a-bd7a-bde4f6917261 None admin] PATH_INFO: `/v3/projects/a4199bd51bd44961a9b4ce3dd6b64f66` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:41.930288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78827b76-2373-481a-bd7a-bde4f6917261 None admin] RBAC: Authorizing `identity:get_project(project_id=a4199bd51bd44961a9b4ce3dd6b64f66)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:41.931712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78827b76-2373-481a-bd7a-bde4f6917261 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:41.933334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 597/1199] 10.222.0.100 () {56 vars in 1191 bytes} [Thu Sep 1 17:25:41 2022] GET /identity/v3/projects/a4199bd51bd44961a9b4ce3dd6b64f66 => generated 362 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:41.939484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38ba61-ff30-47b5-9657-02c9d591ee94 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:25:42.036386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38ba61-ff30-47b5-9657-02c9d591ee94 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:25:42.069047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f0fc0-90ec-4561-983c-c748410919ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:25:42.069678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f0fc0-90ec-4561-983c-c748410919ac None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:25:42.069957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f0fc0-90ec-4561-983c-c748410919ac None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:25:42.070229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f0fc0-90ec-4561-983c-c748410919ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:25:42.070944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f0fc0-90ec-4561-983c-c748410919ac None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=a4199bd51bd44961a9b4ce3dd6b64f66, user.id=4fc023aee4214fcb871bea92003ba310) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:25:42.071254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f0fc0-90ec-4561-983c-c748410919ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:25:42.072679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f0fc0-90ec-4561-983c-c748410919ac None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:25:42.078289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 603/1200] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:25:41 2022] GET /identity/v3/role_assignments?user.id=4fc023aee4214fcb871bea92003ba310&scope.project.id=a4199bd51bd44961a9b4ce3dd6b64f66&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 610 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:25:51.923739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=21 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:25:52.075814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:38.296040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fd563-e83f-4fc4-ac29-7f80a2f64142 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:38.296675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fd563-e83f-4fc4-ac29-7f80a2f64142 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:38.296675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fd563-e83f-4fc4-ac29-7f80a2f64142 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:38.298438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 598/1201] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:26:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:26:38.303675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed90cf-e789-4f33-8cec-7fc105d9a34b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:38.304212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed90cf-e789-4f33-8cec-7fc105d9a34b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:38.304212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed90cf-e789-4f33-8cec-7fc105d9a34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:38.332871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d7ed90cf-e789-4f33-8cec-7fc105d9a34b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:26:38.332871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d7ed90cf-e789-4f33-8cec-7fc105d9a34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:26:38.424804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 604/1202] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:26:38 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:26:38.698280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fd563-e83f-4fc4-ac29-7f80a2f64142 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:38.833611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fd563-e83f-4fc4-ac29-7f80a2f64142 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:38.883475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5c341-ce36-43f4-9332-0b0b71c458e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:38.884213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5c341-ce36-43f4-9332-0b0b71c458e0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:38.884488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5c341-ce36-43f4-9332-0b0b71c458e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:38.884758 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5c341-ce36-43f4-9332-0b0b71c458e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:38.901059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5c341-ce36-43f4-9332-0b0b71c458e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:38.903091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5c341-ce36-43f4-9332-0b0b71c458e0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-97d5c341-ce36-43f4-9332-0b0b71c458e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:26:38.933137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:38.934242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:26:38.935651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:26:38.935651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:38.935651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:26:38.935651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 599/1203] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:38.941712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed90cf-e789-4f33-8cec-7fc105d9a34b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:39.077349 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed90cf-e789-4f33-8cec-7fc105d9a34b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:39.114455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:39.124709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faef3b6-9717-474e-9743-ed14bd6340bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:39.125481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faef3b6-9717-474e-9743-ed14bd6340bd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:39.125763 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faef3b6-9717-474e-9743-ed14bd6340bd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:39.126222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faef3b6-9717-474e-9743-ed14bd6340bd None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:39.127162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faef3b6-9717-474e-9743-ed14bd6340bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:39.127546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faef3b6-9717-474e-9743-ed14bd6340bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:39.130212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faef3b6-9717-474e-9743-ed14bd6340bd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:39.155768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 605/1204] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:26:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:39.163229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5c341-ce36-43f4-9332-0b0b71c458e0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:39.231052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:39.312401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5c341-ce36-43f4-9332-0b0b71c458e0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:39.357038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069b974-b30e-468f-9077-2e8b73ddf185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:39.357796 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069b974-b30e-468f-9077-2e8b73ddf185 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:39.358066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069b974-b30e-468f-9077-2e8b73ddf185 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:39.358341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069b974-b30e-468f-9077-2e8b73ddf185 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:39.359090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069b974-b30e-468f-9077-2e8b73ddf185 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:39.360759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069b974-b30e-468f-9077-2e8b73ddf185 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:39.397848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 600/1205] 10.222.0.100 () {60 vars in 1137 bytes} [Thu Sep 1 17:26:39 2022] POST /identity/v3/users => generated 252 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:26:41.517809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34b6e4-19d3-4693-8755-9ea2d80798e7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:41.518432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34b6e4-19d3-4693-8755-9ea2d80798e7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:41.518537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34b6e4-19d3-4693-8755-9ea2d80798e7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:41.520786 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 606/1206] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:26:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:26:41.526883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111ab5a-c22d-42e5-b47f-d879652955e4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:41.527442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111ab5a-c22d-42e5-b47f-d879652955e4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:41.527613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111ab5a-c22d-42e5-b47f-d879652955e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:41.548042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4111ab5a-c22d-42e5-b47f-d879652955e4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:26:41.549334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4111ab5a-c22d-42e5-b47f-d879652955e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:26:41.628577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 601/1207] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:26:41 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:26:41.912541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34b6e4-19d3-4693-8755-9ea2d80798e7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:42.050615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34b6e4-19d3-4693-8755-9ea2d80798e7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:42.095138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abfd35-0ce9-4f59-a3ba-d5b2afec7f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:42.095963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abfd35-0ce9-4f59-a3ba-d5b2afec7f1b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:42.096301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abfd35-0ce9-4f59-a3ba-d5b2afec7f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:42.096663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abfd35-0ce9-4f59-a3ba-d5b2afec7f1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:42.110008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abfd35-0ce9-4f59-a3ba-d5b2afec7f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e7abfd35-0ce9-4f59-a3ba-d5b2afec7f1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:26:42.116583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:42.117709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:42.118581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 607/1208] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:26:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:42.124517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111ab5a-c22d-42e5-b47f-d879652955e4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:42.170348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:42.239167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111ab5a-c22d-42e5-b47f-d879652955e4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:42.281949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c34b15-1852-4f8c-9b76-d0191bf3bf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:42.282664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c34b15-1852-4f8c-9b76-d0191bf3bf4c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:42.282883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c34b15-1852-4f8c-9b76-d0191bf3bf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:42.283158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c34b15-1852-4f8c-9b76-d0191bf3bf4c None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:42.283791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c34b15-1852-4f8c-9b76-d0191bf3bf4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:42.284105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c34b15-1852-4f8c-9b76-d0191bf3bf4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:42.285687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c34b15-1852-4f8c-9b76-d0191bf3bf4c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:42.295323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 602/1209] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:26:42 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:42.303363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abfd35-0ce9-4f59-a3ba-d5b2afec7f1b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:42.361991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:42.422965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abfd35-0ce9-4f59-a3ba-d5b2afec7f1b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:42.466046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6bc7a-0343-49f7-a02f-1d17db16693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:42.467152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6bc7a-0343-49f7-a02f-1d17db16693f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:42.467152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6bc7a-0343-49f7-a02f-1d17db16693f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:42.467382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6bc7a-0343-49f7-a02f-1d17db16693f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:42.479876 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6bc7a-0343-49f7-a02f-1d17db16693f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:42.481352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6bc7a-0343-49f7-a02f-1d17db16693f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c8a6bc7a-0343-49f7-a02f-1d17db16693f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:26:42.498121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:42.504953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:26:42.506178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:26:42.506178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:42.506178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:42.506178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 608/1210] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:42.506476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c34b15-1852-4f8c-9b76-d0191bf3bf4c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:42.625339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:42.629317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c34b15-1852-4f8c-9b76-d0191bf3bf4c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:42.673334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1373265-0c1d-453a-b124-ff3439fb70ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:42.673933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1373265-0c1d-453a-b124-ff3439fb70ba None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:42.674309 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1373265-0c1d-453a-b124-ff3439fb70ba None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:42.674701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1373265-0c1d-453a-b124-ff3439fb70ba None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:42.676035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1373265-0c1d-453a-b124-ff3439fb70ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:42.676436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1373265-0c1d-453a-b124-ff3439fb70ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:42.677963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1373265-0c1d-453a-b124-ff3439fb70ba None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:42.696282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 603/1211] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:26:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:42.702662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6bc7a-0343-49f7-a02f-1d17db16693f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:42.823143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6bc7a-0343-49f7-a02f-1d17db16693f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:42.824942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:42.867414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270bfa8-162b-47f5-b483-e155dd75d4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:42.868047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270bfa8-162b-47f5-b483-e155dd75d4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:42.868328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270bfa8-162b-47f5-b483-e155dd75d4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:42.868601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270bfa8-162b-47f5-b483-e155dd75d4b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:42.877121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270bfa8-162b-47f5-b483-e155dd75d4b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:42.880017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270bfa8-162b-47f5-b483-e155dd75d4b1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4270bfa8-162b-47f5-b483-e155dd75d4b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:26:42.893854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:42.895050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:42.896038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 609/1212] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:42.901459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1373265-0c1d-453a-b124-ff3439fb70ba None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:43.021662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1373265-0c1d-453a-b124-ff3439fb70ba None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:43.062789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f06407-9054-409b-8a35-ca0bcd015408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:43.063360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f06407-9054-409b-8a35-ca0bcd015408 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:43.063718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f06407-9054-409b-8a35-ca0bcd015408 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:43.063936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f06407-9054-409b-8a35-ca0bcd015408 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:43.064713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f06407-9054-409b-8a35-ca0bcd015408 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:43.065024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f06407-9054-409b-8a35-ca0bcd015408 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:43.066884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f06407-9054-409b-8a35-ca0bcd015408 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:43.093379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 604/1213] 10.222.0.100 () {56 vars in 1148 bytes} [Thu Sep 1 17:26:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:43.094705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270bfa8-162b-47f5-b483-e155dd75d4b1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:43.224143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270bfa8-162b-47f5-b483-e155dd75d4b1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:43.261459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731a679-fc07-441d-9098-599ba8710071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:43.262221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731a679-fc07-441d-9098-599ba8710071 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:43.262561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731a679-fc07-441d-9098-599ba8710071 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:43.262866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731a679-fc07-441d-9098-599ba8710071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:43.296097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:43.339078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731a679-fc07-441d-9098-599ba8710071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:43.340619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731a679-fc07-441d-9098-599ba8710071 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:43.451219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 610/1214] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:26:43 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:26:43.457733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f06407-9054-409b-8a35-ca0bcd015408 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:43.480290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:43.575154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f06407-9054-409b-8a35-ca0bcd015408 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:43.615730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92372b-8aab-440f-8d1e-a1d0c6cbff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:43.616532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92372b-8aab-440f-8d1e-a1d0c6cbff8e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:43.616849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92372b-8aab-440f-8d1e-a1d0c6cbff8e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:43.617186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92372b-8aab-440f-8d1e-a1d0c6cbff8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:43.629272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92372b-8aab-440f-8d1e-a1d0c6cbff8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:43.630979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92372b-8aab-440f-8d1e-a1d0c6cbff8e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4e92372b-8aab-440f-8d1e-a1d0c6cbff8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:26:43.647111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:43.648437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:26:43.649420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:26:43.649420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:43.649420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:26:43.649420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 605/1215] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:43.654689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731a679-fc07-441d-9098-599ba8710071 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:43.724060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:43.766962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731a679-fc07-441d-9098-599ba8710071 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:43.805958 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1c42-1921-4830-960d-7493fee980b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:43.806745 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1c42-1921-4830-960d-7493fee980b0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:43.806925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1c42-1921-4830-960d-7493fee980b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:43.807237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1c42-1921-4830-960d-7493fee980b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:43.807941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1c42-1921-4830-960d-7493fee980b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:43.808294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1c42-1921-4830-960d-7493fee980b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:43.809744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1c42-1921-4830-960d-7493fee980b0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:43.823680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 611/1216] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:26:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:43.829954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92372b-8aab-440f-8d1e-a1d0c6cbff8e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:43.907393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:43.942290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92372b-8aab-440f-8d1e-a1d0c6cbff8e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:43.984003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707204b1-8986-4e3b-a5be-7cb758c87fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:43.984659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707204b1-8986-4e3b-a5be-7cb758c87fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:43.984938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707204b1-8986-4e3b-a5be-7cb758c87fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:43.985208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707204b1-8986-4e3b-a5be-7cb758c87fe7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:43.994319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707204b1-8986-4e3b-a5be-7cb758c87fe7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:43.996533 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707204b1-8986-4e3b-a5be-7cb758c87fe7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-707204b1-8986-4e3b-a5be-7cb758c87fe7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:26:44.010054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:44.011202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:44.012350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:26:44.012350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:26:44.012350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:26:44.012350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:26:44.012350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:26:44.012350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:26:44.012350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:26:44.012350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:26:44.013563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 606/1217] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:26:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:44.020716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1c42-1921-4830-960d-7493fee980b0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:44.135200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1c42-1921-4830-960d-7493fee980b0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:44.166971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:44.185878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fc69d-bcd2-49ba-a03f-964d1e048a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:44.186529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fc69d-bcd2-49ba-a03f-964d1e048a41 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:44.186795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fc69d-bcd2-49ba-a03f-964d1e048a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:44.187058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fc69d-bcd2-49ba-a03f-964d1e048a41 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:44.187709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fc69d-bcd2-49ba-a03f-964d1e048a41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:44.188186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fc69d-bcd2-49ba-a03f-964d1e048a41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:44.189434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fc69d-bcd2-49ba-a03f-964d1e048a41 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:44.208492 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 612/1218] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:26:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:44.209847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707204b1-8986-4e3b-a5be-7cb758c87fe7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:44.325149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707204b1-8986-4e3b-a5be-7cb758c87fe7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:44.359345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:44.367306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2819ea-1bc2-4e52-b097-8115d90c94ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:44.368032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2819ea-1bc2-4e52-b097-8115d90c94ab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:44.368404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2819ea-1bc2-4e52-b097-8115d90c94ab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:44.368770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2819ea-1bc2-4e52-b097-8115d90c94ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:44.369604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2819ea-1bc2-4e52-b097-8115d90c94ab None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=4b181d38588d44d4bb72fa319d2ba2f2) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:44.370032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2819ea-1bc2-4e52-b097-8115d90c94ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:44.371551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2819ea-1bc2-4e52-b097-8115d90c94ab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:44.377346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 607/1219] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:26:44 2022] GET /identity/v3/role_assignments?user.id=4b181d38588d44d4bb72fa319d2ba2f2&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:46.448263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a183af5-bacd-4112-8895-1756f4d86376 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:46.448834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a183af5-bacd-4112-8895-1756f4d86376 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:46.448834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a183af5-bacd-4112-8895-1756f4d86376 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:46.450248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 613/1220] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:26:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:26:46.455690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0ba79-1e51-495c-a523-3ba7ba172b48 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:46.456154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0ba79-1e51-495c-a523-3ba7ba172b48 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:46.456306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0ba79-1e51-495c-a523-3ba7ba172b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:46.474954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-eca0ba79-1e51-495c-a523-3ba7ba172b48 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:26:46.476159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eca0ba79-1e51-495c-a523-3ba7ba172b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:26:46.549259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 608/1221] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:26:46 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:26:46.843285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a183af5-bacd-4112-8895-1756f4d86376 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:46.957462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a183af5-bacd-4112-8895-1756f4d86376 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:47.004221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee47ed-f437-41bb-ad21-72ba787b765d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:47.004861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee47ed-f437-41bb-ad21-72ba787b765d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:47.005141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee47ed-f437-41bb-ad21-72ba787b765d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:47.005412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee47ed-f437-41bb-ad21-72ba787b765d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:47.022366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee47ed-f437-41bb-ad21-72ba787b765d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-05ee47ed-f437-41bb-ad21-72ba787b765d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:26:47.027149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:47.028495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:47.029490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 614/1222] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:26:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:47.037608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0ba79-1e51-495c-a523-3ba7ba172b48 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:47.142911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0ba79-1e51-495c-a523-3ba7ba172b48 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:47.169112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:47.182177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7994d-cac1-4de7-92d0-a124fb28f425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:47.182770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7994d-cac1-4de7-92d0-a124fb28f425 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:47.183060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7994d-cac1-4de7-92d0-a124fb28f425 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:47.183311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7994d-cac1-4de7-92d0-a124fb28f425 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:47.183976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7994d-cac1-4de7-92d0-a124fb28f425 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:47.184279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7994d-cac1-4de7-92d0-a124fb28f425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:47.185916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7994d-cac1-4de7-92d0-a124fb28f425 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:47.197152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 609/1223] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:26:47 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:47.203594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee47ed-f437-41bb-ad21-72ba787b765d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:47.250539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:47.313815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee47ed-f437-41bb-ad21-72ba787b765d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:47.351285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d32c3-544c-485e-b0c3-8f6d0a9f6e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:47.352005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d32c3-544c-485e-b0c3-8f6d0a9f6e2f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:47.352223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d32c3-544c-485e-b0c3-8f6d0a9f6e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:47.352496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d32c3-544c-485e-b0c3-8f6d0a9f6e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:47.363778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d32c3-544c-485e-b0c3-8f6d0a9f6e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:47.365280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d32c3-544c-485e-b0c3-8f6d0a9f6e2f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-cb0d32c3-544c-485e-b0c3-8f6d0a9f6e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:26:47.381192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:47.382528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:26:47.383917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:26:47.383917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:47.383917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:47.383917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 615/1224] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:26:47.388738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7994d-cac1-4de7-92d0-a124fb28f425 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:47.529760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7994d-cac1-4de7-92d0-a124fb28f425 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:47.581988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c5c5a-c2a8-4596-a5eb-cc254701731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:47.582770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c5c5a-c2a8-4596-a5eb-cc254701731f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:47.583161 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c5c5a-c2a8-4596-a5eb-cc254701731f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:47.583610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c5c5a-c2a8-4596-a5eb-cc254701731f None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:47.584623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c5c5a-c2a8-4596-a5eb-cc254701731f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:47.585192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c5c5a-c2a8-4596-a5eb-cc254701731f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:47.587657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c5c5a-c2a8-4596-a5eb-cc254701731f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:47.610906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 610/1225] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:26:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:26:47.619215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d32c3-544c-485e-b0c3-8f6d0a9f6e2f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:47.740700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:47.748480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d32c3-544c-485e-b0c3-8f6d0a9f6e2f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:47.789119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed31a47-8525-4c5b-83b4-abed6e7e479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:47.789619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31a47-8525-4c5b-83b4-abed6e7e479e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:47.789985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31a47-8525-4c5b-83b4-abed6e7e479e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:47.790267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31a47-8525-4c5b-83b4-abed6e7e479e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:47.798624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed31a47-8525-4c5b-83b4-abed6e7e479e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:47.800677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed31a47-8525-4c5b-83b4-abed6e7e479e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7ed31a47-8525-4c5b-83b4-abed6e7e479e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:26:47.812911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:47.814186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:47.816121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 616/1226] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:47.820899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c5c5a-c2a8-4596-a5eb-cc254701731f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:47.874175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:47.945883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c5c5a-c2a8-4596-a5eb-cc254701731f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:47.993145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6e67-fb0a-46be-be66-4ea161bd7fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:47.994439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6e67-fb0a-46be-be66-4ea161bd7fbd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:47.994439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6e67-fb0a-46be-be66-4ea161bd7fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:47.994439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6e67-fb0a-46be-be66-4ea161bd7fbd None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:47.995342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f6e67-fb0a-46be-be66-4ea161bd7fbd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:47.995435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f6e67-fb0a-46be-be66-4ea161bd7fbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:47.996884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f6e67-fb0a-46be-be66-4ea161bd7fbd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:48.023413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 611/1227] 10.222.0.100 () {56 vars in 1148 bytes} [Thu Sep 1 17:26:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:48.029921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed31a47-8525-4c5b-83b4-abed6e7e479e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:48.164484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed31a47-8525-4c5b-83b4-abed6e7e479e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:48.208253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105213f-26ab-4244-9e4c-57e43c459184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:48.208908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105213f-26ab-4244-9e4c-57e43c459184 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:48.209219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105213f-26ab-4244-9e4c-57e43c459184 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:48.209477 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105213f-26ab-4244-9e4c-57e43c459184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:48.224774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105213f-26ab-4244-9e4c-57e43c459184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:48.226494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105213f-26ab-4244-9e4c-57e43c459184 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9105213f-26ab-4244-9e4c-57e43c459184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:26:48.246216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:48.249671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:26:48.250959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:26:48.250959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:48.250959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:48.250959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 617/1228] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:48.254149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6e67-fb0a-46be-be66-4ea161bd7fbd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:48.380207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6e67-fb0a-46be-be66-4ea161bd7fbd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:48.383093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:48.422343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852189f9-922c-447d-9fd8-0d3aaa423c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:48.422888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852189f9-922c-447d-9fd8-0d3aaa423c77 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:48.423163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852189f9-922c-447d-9fd8-0d3aaa423c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:48.423437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852189f9-922c-447d-9fd8-0d3aaa423c77 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:48.424100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852189f9-922c-447d-9fd8-0d3aaa423c77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:48.424423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852189f9-922c-447d-9fd8-0d3aaa423c77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:48.426274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852189f9-922c-447d-9fd8-0d3aaa423c77 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:48.445019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 612/1229] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:26:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:26:48.452380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105213f-26ab-4244-9e4c-57e43c459184 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:48.465346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:48.576419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105213f-26ab-4244-9e4c-57e43c459184 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:48.615225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7c57f-a87b-4ab5-8d3d-0d2d72106c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:48.615871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7c57f-a87b-4ab5-8d3d-0d2d72106c51 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:48.616152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7c57f-a87b-4ab5-8d3d-0d2d72106c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:48.616426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7c57f-a87b-4ab5-8d3d-0d2d72106c51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:48.624026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7c57f-a87b-4ab5-8d3d-0d2d72106c51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:48.625513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7c57f-a87b-4ab5-8d3d-0d2d72106c51 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f8d7c57f-a87b-4ab5-8d3d-0d2d72106c51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:26:48.640750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:48.642138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:48.643773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:26:48.643773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:26:48.643773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:26:48.643773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:26:48.643773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:26:48.643773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:26:48.643773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:26:48.643773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:48.643773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 618/1230] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:26:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:26:48.648632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852189f9-922c-447d-9fd8-0d3aaa423c77 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:48.770411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852189f9-922c-447d-9fd8-0d3aaa423c77 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:48.819350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72631ce1-832f-4951-b241-f44c35858e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:48.819969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72631ce1-832f-4951-b241-f44c35858e1e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:48.820279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72631ce1-832f-4951-b241-f44c35858e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:48.820517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72631ce1-832f-4951-b241-f44c35858e1e None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:48.821232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72631ce1-832f-4951-b241-f44c35858e1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:48.821518 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72631ce1-832f-4951-b241-f44c35858e1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:48.822978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72631ce1-832f-4951-b241-f44c35858e1e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:48.836471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 613/1231] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:26:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:48.846028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7c57f-a87b-4ab5-8d3d-0d2d72106c51 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:48.928237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:48.969128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7c57f-a87b-4ab5-8d3d-0d2d72106c51 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:49.010133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5bc83-c90e-4708-ac7d-84bc6eb75f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:49.010783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5bc83-c90e-4708-ac7d-84bc6eb75f09 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:49.011161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5bc83-c90e-4708-ac7d-84bc6eb75f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:49.011362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5bc83-c90e-4708-ac7d-84bc6eb75f09 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/4b181d38588d44d4bb72fa319d2ba2f2/roles/3e624c5fdefb4c03bd915bbc9d097212` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:49.027626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5bc83-c90e-4708-ac7d-84bc6eb75f09 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=4b181d38588d44d4bb72fa319d2ba2f2, role_id=3e624c5fdefb4c03bd915bbc9d097212)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:49.029022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5bc83-c90e-4708-ac7d-84bc6eb75f09 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:49.044504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 619/1232] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:26:48 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/4b181d38588d44d4bb72fa319d2ba2f2/roles/3e624c5fdefb4c03bd915bbc9d097212 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:26:51.202808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2b9e3-0824-4836-91f5-95d736be850a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:51.203369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2b9e3-0824-4836-91f5-95d736be850a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:51.203369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2b9e3-0824-4836-91f5-95d736be850a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:51.204664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 614/1233] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:26:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:26:51.210506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47126518-d9e6-4f9d-9416-928e45a8014c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:51.211160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47126518-d9e6-4f9d-9416-928e45a8014c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:51.211160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47126518-d9e6-4f9d-9416-928e45a8014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:51.229854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-47126518-d9e6-4f9d-9416-928e45a8014c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:26:51.231054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-47126518-d9e6-4f9d-9416-928e45a8014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:26:51.305409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 620/1234] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:26:51 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:26:51.580013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2b9e3-0824-4836-91f5-95d736be850a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:51.608897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:51.698211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2b9e3-0824-4836-91f5-95d736be850a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:51.734751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba406839-b95e-4e39-9867-1dc52215ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:51.735395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba406839-b95e-4e39-9867-1dc52215ef62 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:51.735673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba406839-b95e-4e39-9867-1dc52215ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:51.735976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba406839-b95e-4e39-9867-1dc52215ef62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:51.747771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba406839-b95e-4e39-9867-1dc52215ef62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-ba406839-b95e-4e39-9867-1dc52215ef62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:26:51.753665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:51.754992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:26:51.755856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 615/1235] 10.222.0.100 () {56 vars in 1107 bytes} [Thu Sep 1 17:26:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:51.761596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47126518-d9e6-4f9d-9416-928e45a8014c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:51.812868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:51.874707 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47126518-d9e6-4f9d-9416-928e45a8014c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:51.913532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8ea87-0b1d-470a-b043-94065136cf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:51.914183 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8ea87-0b1d-470a-b043-94065136cf08 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:51.914467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8ea87-0b1d-470a-b043-94065136cf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:51.914738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8ea87-0b1d-470a-b043-94065136cf08 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:51.915412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8ea87-0b1d-470a-b043-94065136cf08 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:51.915717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8ea87-0b1d-470a-b043-94065136cf08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:51.917136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8ea87-0b1d-470a-b043-94065136cf08 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:51.929604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 621/1236] 10.222.0.100 () {56 vars in 1108 bytes} [Thu Sep 1 17:26:51 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:51.932957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba406839-b95e-4e39-9867-1dc52215ef62 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:52.047619 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:52.064633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba406839-b95e-4e39-9867-1dc52215ef62 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:52.100615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef0ce-c8ab-4ebd-9cf2-2c4814eeb620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:52.101248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef0ce-c8ab-4ebd-9cf2-2c4814eeb620 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:52.101528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef0ce-c8ab-4ebd-9cf2-2c4814eeb620 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:52.101828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef0ce-c8ab-4ebd-9cf2-2c4814eeb620 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:52.112052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bef0ce-c8ab-4ebd-9cf2-2c4814eeb620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:52.114287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bef0ce-c8ab-4ebd-9cf2-2c4814eeb620 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f2bef0ce-c8ab-4ebd-9cf2-2c4814eeb620 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:26:52.129756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:52.130913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:26:52.132165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:26:52.132165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:52.132165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:26:52.132165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 616/1237] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:52.137114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8ea87-0b1d-470a-b043-94065136cf08 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:52.251025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8ea87-0b1d-470a-b043-94065136cf08 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:52.261047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:52.291297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7430672-a2f5-45e5-8823-ba3bedee3575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:52.291961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7430672-a2f5-45e5-8823-ba3bedee3575 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:52.292273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7430672-a2f5-45e5-8823-ba3bedee3575 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:52.292589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7430672-a2f5-45e5-8823-ba3bedee3575 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:52.293273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7430672-a2f5-45e5-8823-ba3bedee3575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:52.293604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7430672-a2f5-45e5-8823-ba3bedee3575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:52.295094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7430672-a2f5-45e5-8823-ba3bedee3575 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:52.308249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 622/1238] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:26:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:52.315135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef0ce-c8ab-4ebd-9cf2-2c4814eeb620 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:52.434833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef0ce-c8ab-4ebd-9cf2-2c4814eeb620 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:52.471234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6178c7-6911-406c-b895-65f8355c6299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:52.472523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6178c7-6911-406c-b895-65f8355c6299 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:52.472804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6178c7-6911-406c-b895-65f8355c6299 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:52.473078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6178c7-6911-406c-b895-65f8355c6299 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:52.475995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:52.481632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6178c7-6911-406c-b895-65f8355c6299 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:52.483237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6178c7-6911-406c-b895-65f8355c6299 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cd6178c7-6911-406c-b895-65f8355c6299 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:26:52.494764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:52.496241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:26:52.497618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 617/1239] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:52.504931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7430672-a2f5-45e5-8823-ba3bedee3575 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:52.618077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7430672-a2f5-45e5-8823-ba3bedee3575 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:52.663462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffd4e0-af32-4448-a5fc-53e0b340fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:52.664093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffd4e0-af32-4448-a5fc-53e0b340fd02 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:52.664371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffd4e0-af32-4448-a5fc-53e0b340fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:52.664638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffd4e0-af32-4448-a5fc-53e0b340fd02 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:52.665492 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffd4e0-af32-4448-a5fc-53e0b340fd02 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:52.665789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffd4e0-af32-4448-a5fc-53e0b340fd02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:52.667594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffd4e0-af32-4448-a5fc-53e0b340fd02 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:52.687508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 623/1240] 10.222.0.100 () {56 vars in 1148 bytes} [Thu Sep 1 17:26:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:26:52.693809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6178c7-6911-406c-b895-65f8355c6299 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:52.819969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6178c7-6911-406c-b895-65f8355c6299 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:52.864775 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7863c-4618-463a-bf7f-f5b41895aa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:52.865368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7863c-4618-463a-bf7f-f5b41895aa8a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:52.865678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7863c-4618-463a-bf7f-f5b41895aa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:52.865960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7863c-4618-463a-bf7f-f5b41895aa8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:52.938937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:52.948019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7863c-4618-463a-bf7f-f5b41895aa8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:52.949550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7863c-4618-463a-bf7f-f5b41895aa8a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:53.081625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 618/1241] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:26:52 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:26:53.087470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffd4e0-af32-4448-a5fc-53e0b340fd02 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:53.119917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:53.206711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffd4e0-af32-4448-a5fc-53e0b340fd02 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:53.244224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9a6a0-1c2c-4043-b42d-443756177bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:53.245123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9a6a0-1c2c-4043-b42d-443756177bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:53.245515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9a6a0-1c2c-4043-b42d-443756177bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:53.245894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9a6a0-1c2c-4043-b42d-443756177bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:53.258813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9a6a0-1c2c-4043-b42d-443756177bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:53.260290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9a6a0-1c2c-4043-b42d-443756177bc2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5be9a6a0-1c2c-4043-b42d-443756177bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:26:53.276916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:53.278128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:26:53.279151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:26:53.279151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:53.279151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:53.279151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 624/1242] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:53.284197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7863c-4618-463a-bf7f-f5b41895aa8a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:53.388713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:53.395916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7863c-4618-463a-bf7f-f5b41895aa8a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:53.432847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc3285-00a8-496a-918b-3761c550592c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:53.433454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc3285-00a8-496a-918b-3761c550592c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:53.433784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc3285-00a8-496a-918b-3761c550592c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:53.434037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc3285-00a8-496a-918b-3761c550592c None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:53.434677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc3285-00a8-496a-918b-3761c550592c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:53.434984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc3285-00a8-496a-918b-3761c550592c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:53.436343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc3285-00a8-496a-918b-3761c550592c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:53.449724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 619/1243] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:26:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:26:53.455592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9a6a0-1c2c-4043-b42d-443756177bc2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:53.546788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:53.571450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9a6a0-1c2c-4043-b42d-443756177bc2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:53.611994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f17c9-fb3a-4363-ac71-1670560ab525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:53.612877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f17c9-fb3a-4363-ac71-1670560ab525 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:53.613201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f17c9-fb3a-4363-ac71-1670560ab525 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:53.613531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f17c9-fb3a-4363-ac71-1670560ab525 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:53.624056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f17c9-fb3a-4363-ac71-1670560ab525 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:53.625988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f17c9-fb3a-4363-ac71-1670560ab525 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9f4f17c9-fb3a-4363-ac71-1670560ab525 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:26:53.639072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:53.640228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:53.641372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:26:53.641372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:26:53.641372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:26:53.641372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:26:53.641372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:26:53.641372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:26:53.641372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:26:53.641372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:53.641372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 625/1244] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:26:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:26:53.646855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc3285-00a8-496a-918b-3761c550592c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:53.765351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc3285-00a8-496a-918b-3761c550592c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:53.807150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf474c-603c-49ab-b417-78f31737d9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:53.807996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf474c-603c-49ab-b417-78f31737d9cc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:53.808425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf474c-603c-49ab-b417-78f31737d9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:53.808756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf474c-603c-49ab-b417-78f31737d9cc None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:53.809632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf474c-603c-49ab-b417-78f31737d9cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:53.809936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf474c-603c-49ab-b417-78f31737d9cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:53.811382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf474c-603c-49ab-b417-78f31737d9cc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:53.826060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 620/1245] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:26:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:53.833276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f17c9-fb3a-4363-ac71-1670560ab525 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:53.952586 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f17c9-fb3a-4363-ac71-1670560ab525 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:53.992884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11f4c6-8b5b-48d0-aed3-eed9f6f00d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:53.993897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f4c6-8b5b-48d0-aed3-eed9f6f00d4f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:53.994076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f4c6-8b5b-48d0-aed3-eed9f6f00d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:53.994332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f4c6-8b5b-48d0-aed3-eed9f6f00d4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:53.995070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11f4c6-8b5b-48d0-aed3-eed9f6f00d4f None admin] RBAC: Adding query filter params (role.id=3e624c5fdefb4c03bd915bbc9d097212, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=4b181d38588d44d4bb72fa319d2ba2f2) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:53.995403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11f4c6-8b5b-48d0-aed3-eed9f6f00d4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:53.996808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11f4c6-8b5b-48d0-aed3-eed9f6f00d4f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:54.003463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 626/1246] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:26:53 2022] GET /identity/v3/role_assignments?user.id=4b181d38588d44d4bb72fa319d2ba2f2&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=3e624c5fdefb4c03bd915bbc9d097212 => generated 610 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:56.349998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149f04d-c25e-4c7c-a811-a20fb117546d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:56.350572 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149f04d-c25e-4c7c-a811-a20fb117546d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:56.350572 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149f04d-c25e-4c7c-a811-a20fb117546d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:56.351887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 621/1247] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:26:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:26:56.359784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39396-098c-4dcb-ae0a-da96151b7ae3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:56.360253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39396-098c-4dcb-ae0a-da96151b7ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:56.360345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39396-098c-4dcb-ae0a-da96151b7ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:56.367335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:56.382092 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c7d39396-098c-4dcb-ae0a-da96151b7ae3 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:26:56.383438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c7d39396-098c-4dcb-ae0a-da96151b7ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:26:56.461284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 627/1248] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:26:56 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:26:56.767008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149f04d-c25e-4c7c-a811-a20fb117546d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:56.790629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:56.884033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149f04d-c25e-4c7c-a811-a20fb117546d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:56.928332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51e9f5-0011-4dee-917f-d51774b45fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:56.928915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51e9f5-0011-4dee-917f-d51774b45fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:56.929248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51e9f5-0011-4dee-917f-d51774b45fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:56.929519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51e9f5-0011-4dee-917f-d51774b45fb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:56.941177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51e9f5-0011-4dee-917f-d51774b45fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9a51e9f5-0011-4dee-917f-d51774b45fb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:26:56.946384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:56.947375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:26:56.948508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 622/1249] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:26:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:56.954451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d39396-098c-4dcb-ae0a-da96151b7ae3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:57.078652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d39396-098c-4dcb-ae0a-da96151b7ae3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:57.114235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:57.120723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d10d31-f014-4ec1-b413-aafee7f503e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:57.121348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d10d31-f014-4ec1-b413-aafee7f503e6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:57.121657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d10d31-f014-4ec1-b413-aafee7f503e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:57.121941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d10d31-f014-4ec1-b413-aafee7f503e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:57.122601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d10d31-f014-4ec1-b413-aafee7f503e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:57.122906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d10d31-f014-4ec1-b413-aafee7f503e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:57.124405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d10d31-f014-4ec1-b413-aafee7f503e6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:57.133910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 628/1250] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:26:56 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:57.139980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51e9f5-0011-4dee-917f-d51774b45fb0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:57.227279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:57.253834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51e9f5-0011-4dee-917f-d51774b45fb0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:57.296389 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644246e3-e53f-489e-a3dc-64194e526feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:57.297386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644246e3-e53f-489e-a3dc-64194e526feb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:57.297885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644246e3-e53f-489e-a3dc-64194e526feb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:57.298339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644246e3-e53f-489e-a3dc-64194e526feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:57.317810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644246e3-e53f-489e-a3dc-64194e526feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:57.319300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644246e3-e53f-489e-a3dc-64194e526feb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-644246e3-e53f-489e-a3dc-64194e526feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:26:57.337242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:57.339224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:26:57.341142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:26:57.341142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:57.341142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:26:57.341142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 623/1251] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:57.344586 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d10d31-f014-4ec1-b413-aafee7f503e6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:57.462080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d10d31-f014-4ec1-b413-aafee7f503e6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:57.498184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be19415-b6ef-4743-80cc-78d6803b8139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:57.498712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be19415-b6ef-4743-80cc-78d6803b8139 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:57.498990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be19415-b6ef-4743-80cc-78d6803b8139 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:57.499349 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be19415-b6ef-4743-80cc-78d6803b8139 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:57.499998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be19415-b6ef-4743-80cc-78d6803b8139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:57.500297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be19415-b6ef-4743-80cc-78d6803b8139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:57.501788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be19415-b6ef-4743-80cc-78d6803b8139 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:57.514585 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 629/1252] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:26:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:57.522102 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644246e3-e53f-489e-a3dc-64194e526feb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:57.647094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644246e3-e53f-489e-a3dc-64194e526feb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:57.691290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:57.700505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689db768-1e26-4813-ad2f-25b19c56af2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:57.701276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689db768-1e26-4813-ad2f-25b19c56af2d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:57.701552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689db768-1e26-4813-ad2f-25b19c56af2d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:57.701886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689db768-1e26-4813-ad2f-25b19c56af2d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:57.711380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689db768-1e26-4813-ad2f-25b19c56af2d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:57.712949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689db768-1e26-4813-ad2f-25b19c56af2d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-689db768-1e26-4813-ad2f-25b19c56af2d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:26:57.727256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:57.728495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:26:57.729609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 624/1253] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:57.734939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be19415-b6ef-4743-80cc-78d6803b8139 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:57.800126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:57.853604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be19415-b6ef-4743-80cc-78d6803b8139 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:57.910893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebd683-854b-48c8-b123-e4b09860db28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:57.911517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebd683-854b-48c8-b123-e4b09860db28 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:57.911792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebd683-854b-48c8-b123-e4b09860db28 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:57.912064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebd683-854b-48c8-b123-e4b09860db28 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:57.912806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebd683-854b-48c8-b123-e4b09860db28 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:57.913111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebd683-854b-48c8-b123-e4b09860db28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:57.914584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebd683-854b-48c8-b123-e4b09860db28 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:57.934016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 630/1254] 10.222.0.100 () {56 vars in 1148 bytes} [Thu Sep 1 17:26:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:57.940078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689db768-1e26-4813-ad2f-25b19c56af2d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:58.057782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689db768-1e26-4813-ad2f-25b19c56af2d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:58.095763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0d310-a880-4140-a1c3-4721792374c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:58.096380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0d310-a880-4140-a1c3-4721792374c7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:58.096738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0d310-a880-4140-a1c3-4721792374c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:58.097006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0d310-a880-4140-a1c3-4721792374c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:58.156603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:58.169608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0d310-a880-4140-a1c3-4721792374c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:58.171058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0d310-a880-4140-a1c3-4721792374c7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:58.287938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 625/1255] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:26:57 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:26:58.294248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebd683-854b-48c8-b123-e4b09860db28 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:58.422862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebd683-854b-48c8-b123-e4b09860db28 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:58.441062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:58.473327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c40b4-cdae-4896-8a9a-60069a612c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:58.474020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c40b4-cdae-4896-8a9a-60069a612c2e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:58.474305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c40b4-cdae-4896-8a9a-60069a612c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:58.474564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c40b4-cdae-4896-8a9a-60069a612c2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:58.485918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c40b4-cdae-4896-8a9a-60069a612c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:58.487889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c40b4-cdae-4896-8a9a-60069a612c2e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e51c40b4-cdae-4896-8a9a-60069a612c2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:26:58.507220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:58.508501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:26:58.509611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:26:58.509611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:26:58.509611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:58.509611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 631/1256] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:26:58.515006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0d310-a880-4140-a1c3-4721792374c7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:58.632837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:58.654191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0d310-a880-4140-a1c3-4721792374c7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:58.699879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0c48f-cfff-4e58-bc77-0cec647f5ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:58.700415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0c48f-cfff-4e58-bc77-0cec647f5ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:58.700705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0c48f-cfff-4e58-bc77-0cec647f5ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:58.700969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0c48f-cfff-4e58-bc77-0cec647f5ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:58.701640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0c48f-cfff-4e58-bc77-0cec647f5ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:58.701943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0c48f-cfff-4e58-bc77-0cec647f5ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:58.703469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0c48f-cfff-4e58-bc77-0cec647f5ca6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:58.717302 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 626/1257] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:26:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:58.724268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c40b4-cdae-4896-8a9a-60069a612c2e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:58.855810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c40b4-cdae-4896-8a9a-60069a612c2e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:58.896622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e39c9-9e8b-4419-a0da-fa2839e86161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:58.897273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e39c9-9e8b-4419-a0da-fa2839e86161 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:58.897556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e39c9-9e8b-4419-a0da-fa2839e86161 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:58.897858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e39c9-9e8b-4419-a0da-fa2839e86161 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:58.906410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e39c9-9e8b-4419-a0da-fa2839e86161 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:58.907894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e39c9-9e8b-4419-a0da-fa2839e86161 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e43e39c9-9e8b-4419-a0da-fa2839e86161 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:26:58.922375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:26:58.923935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:26:58.925448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:26:58.925448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:26:58.925448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:26:58.925448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:26:58.925448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:26:58.925448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:26:58.925448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:26:58.925448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:26:58.925448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 632/1258] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:26:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:26:58.929779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0c48f-cfff-4e58-bc77-0cec647f5ca6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:59.053708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0c48f-cfff-4e58-bc77-0cec647f5ca6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:59.089519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9b09a-fcbb-4e4f-a6e8-c647e857dd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:59.090172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9b09a-fcbb-4e4f-a6e8-c647e857dd74 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:59.090450 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9b09a-fcbb-4e4f-a6e8-c647e857dd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:59.090719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9b09a-fcbb-4e4f-a6e8-c647e857dd74 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:59.091427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9b09a-fcbb-4e4f-a6e8-c647e857dd74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:59.091729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9b09a-fcbb-4e4f-a6e8-c647e857dd74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:59.093200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9b09a-fcbb-4e4f-a6e8-c647e857dd74 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:59.107864 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 627/1259] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:26:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:26:59.115389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e39c9-9e8b-4419-a0da-fa2839e86161 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:26:59.145567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:26:59.232002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e39c9-9e8b-4419-a0da-fa2839e86161 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:26:59.267061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f42ddb-4064-4ffb-aeeb-eb893199faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:26:59.267667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f42ddb-4064-4ffb-aeeb-eb893199faf0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:26:59.267953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f42ddb-4064-4ffb-aeeb-eb893199faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:26:59.268224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f42ddb-4064-4ffb-aeeb-eb893199faf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:26:59.269688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f42ddb-4064-4ffb-aeeb-eb893199faf0 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=4b181d38588d44d4bb72fa319d2ba2f2) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:26:59.270003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f42ddb-4064-4ffb-aeeb-eb893199faf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:26:59.271444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f42ddb-4064-4ffb-aeeb-eb893199faf0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:26:59.277337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 633/1260] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:26:59 2022] GET /identity/v3/role_assignments?user.id=4b181d38588d44d4bb72fa319d2ba2f2&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:01.168152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5f2fd-3c56-4733-b103-66cddedebbfd None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:01.168677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5f2fd-3c56-4733-b103-66cddedebbfd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:01.168677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5f2fd-3c56-4733-b103-66cddedebbfd None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:01.170044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 628/1261] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:27:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:27:01.175087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204fb04-8c8a-463f-9000-91bbcd72c72e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:01.175522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204fb04-8c8a-463f-9000-91bbcd72c72e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:01.175617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204fb04-8c8a-463f-9000-91bbcd72c72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:01.194578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8204fb04-8c8a-463f-9000-91bbcd72c72e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:01.195755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8204fb04-8c8a-463f-9000-91bbcd72c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:01.262547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 634/1262] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:27:01 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:27:01.531771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5f2fd-3c56-4733-b103-66cddedebbfd None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:01.542780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:01.636556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5f2fd-3c56-4733-b103-66cddedebbfd None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:01.670624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a81a52-3a79-49d9-834c-910ed5f9ea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:01.671243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a81a52-3a79-49d9-834c-910ed5f9ea64 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:01.671516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a81a52-3a79-49d9-834c-910ed5f9ea64 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:01.671782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a81a52-3a79-49d9-834c-910ed5f9ea64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:01.682692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a81a52-3a79-49d9-834c-910ed5f9ea64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-98a81a52-3a79-49d9-834c-910ed5f9ea64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:27:01.687654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:01.688712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:27:01.689831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 629/1263] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:27:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:01.697658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204fb04-8c8a-463f-9000-91bbcd72c72e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:01.717622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:01.797374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204fb04-8c8a-463f-9000-91bbcd72c72e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:01.831391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1792d-2e8f-4f51-af81-bfdf7ac04d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:01.831983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1792d-2e8f-4f51-af81-bfdf7ac04d73 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:01.832267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1792d-2e8f-4f51-af81-bfdf7ac04d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:01.832539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1792d-2e8f-4f51-af81-bfdf7ac04d73 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:01.833201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1792d-2e8f-4f51-af81-bfdf7ac04d73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:01.833503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1792d-2e8f-4f51-af81-bfdf7ac04d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:01.834895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1792d-2e8f-4f51-af81-bfdf7ac04d73 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:01.842924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 635/1264] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:27:01 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:01.850131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a81a52-3a79-49d9-834c-910ed5f9ea64 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:01.916495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:01.949664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a81a52-3a79-49d9-834c-910ed5f9ea64 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:01.988235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5935c-9afb-4316-bad7-409d6d25d965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:01.989246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5935c-9afb-4316-bad7-409d6d25d965 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:01.989376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5935c-9afb-4316-bad7-409d6d25d965 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:01.989768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5935c-9afb-4316-bad7-409d6d25d965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:02.001716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5935c-9afb-4316-bad7-409d6d25d965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:02.003905 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5935c-9afb-4316-bad7-409d6d25d965 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-edd5935c-9afb-4316-bad7-409d6d25d965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:27:02.021755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:02.022812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:27:02.023906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:27:02.023906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:27:02.023906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:27:02.023906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 630/1265] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:02.029104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1792d-2e8f-4f51-af81-bfdf7ac04d73 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:02.110436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:02.130448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1792d-2e8f-4f51-af81-bfdf7ac04d73 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:02.164027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84229430-e1eb-4cac-8714-65434660f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:02.164623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84229430-e1eb-4cac-8714-65434660f3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:02.164901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84229430-e1eb-4cac-8714-65434660f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:02.165170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84229430-e1eb-4cac-8714-65434660f3d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:02.165830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84229430-e1eb-4cac-8714-65434660f3d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:02.166128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84229430-e1eb-4cac-8714-65434660f3d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:02.167470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84229430-e1eb-4cac-8714-65434660f3d6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:02.178994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 636/1266] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:27:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:02.184821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5935c-9afb-4316-bad7-409d6d25d965 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:02.283329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5935c-9afb-4316-bad7-409d6d25d965 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:02.303228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:02.317146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d870e96-16bc-44a9-a0bb-37241ec89180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:02.317778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d870e96-16bc-44a9-a0bb-37241ec89180 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:02.318055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d870e96-16bc-44a9-a0bb-37241ec89180 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:02.318323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d870e96-16bc-44a9-a0bb-37241ec89180 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:02.325388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d870e96-16bc-44a9-a0bb-37241ec89180 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:02.326829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d870e96-16bc-44a9-a0bb-37241ec89180 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9d870e96-16bc-44a9-a0bb-37241ec89180 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:27:02.338532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:02.339727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:27:02.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 631/1267] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:27:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:02.345690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84229430-e1eb-4cac-8714-65434660f3d6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:02.444597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84229430-e1eb-4cac-8714-65434660f3d6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:02.478508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70322e9b-7412-45ee-9539-0205ec6e1e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:02.479100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70322e9b-7412-45ee-9539-0205ec6e1e64 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:02.479376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70322e9b-7412-45ee-9539-0205ec6e1e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:02.479648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70322e9b-7412-45ee-9539-0205ec6e1e64 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:02.480359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70322e9b-7412-45ee-9539-0205ec6e1e64 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:02.480661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70322e9b-7412-45ee-9539-0205ec6e1e64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:02.482112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70322e9b-7412-45ee-9539-0205ec6e1e64 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:02.490633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:02.499766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 637/1268] 10.222.0.100 () {56 vars in 1148 bytes} [Thu Sep 1 17:27:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:02.506124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d870e96-16bc-44a9-a0bb-37241ec89180 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:02.608829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d870e96-16bc-44a9-a0bb-37241ec89180 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:02.652579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d1d2c-3ebd-4642-9429-64b638f8a929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:02.653202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d1d2c-3ebd-4642-9429-64b638f8a929 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:02.653474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d1d2c-3ebd-4642-9429-64b638f8a929 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:02.653769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d1d2c-3ebd-4642-9429-64b638f8a929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:02.665424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d1d2c-3ebd-4642-9429-64b638f8a929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:02.667241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d1d2c-3ebd-4642-9429-64b638f8a929 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-847d1d2c-3ebd-4642-9429-64b638f8a929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:27:02.681466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:02.682607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:27:02.683870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:27:02.683870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:27:02.683870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:27:02.683870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 632/1269] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:02.688521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70322e9b-7412-45ee-9539-0205ec6e1e64 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:02.786158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70322e9b-7412-45ee-9539-0205ec6e1e64 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:02.827214 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a036d47-c758-42b3-94d4-912591d4b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:02.827818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a036d47-c758-42b3-94d4-912591d4b594 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:02.828101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a036d47-c758-42b3-94d4-912591d4b594 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:02.828373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a036d47-c758-42b3-94d4-912591d4b594 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:02.829020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a036d47-c758-42b3-94d4-912591d4b594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:02.829323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a036d47-c758-42b3-94d4-912591d4b594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:02.830749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a036d47-c758-42b3-94d4-912591d4b594 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:02.842622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 638/1270] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:27:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:02.848373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d1d2c-3ebd-4642-9429-64b638f8a929 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:02.880163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:02.947919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d1d2c-3ebd-4642-9429-64b638f8a929 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:02.983850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d718a-299a-4408-91dd-18c941060941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:02.984486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d718a-299a-4408-91dd-18c941060941 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:02.984807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d718a-299a-4408-91dd-18c941060941 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:02.985080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d718a-299a-4408-91dd-18c941060941 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:02.992635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d718a-299a-4408-91dd-18c941060941 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:02.994093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d718a-299a-4408-91dd-18c941060941 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e67d718a-299a-4408-91dd-18c941060941 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:27:03.005736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:03.006801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:03.007902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:27:03.007902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:27:03.007902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:27:03.007902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:27:03.007902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:27:03.007902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:27:03.007902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:27:03.007902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:27:03.007902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 633/1271] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:27:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:03.012835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a036d47-c758-42b3-94d4-912591d4b594 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:03.070257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:03.110322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a036d47-c758-42b3-94d4-912591d4b594 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:03.144276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749ab2d-fd97-408a-b135-be6cfad36afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:03.144885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749ab2d-fd97-408a-b135-be6cfad36afa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:03.145164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749ab2d-fd97-408a-b135-be6cfad36afa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:03.145440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749ab2d-fd97-408a-b135-be6cfad36afa None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:03.146165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749ab2d-fd97-408a-b135-be6cfad36afa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:03.146470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749ab2d-fd97-408a-b135-be6cfad36afa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:03.147903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749ab2d-fd97-408a-b135-be6cfad36afa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:03.161635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 639/1272] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:03.167952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d718a-299a-4408-91dd-18c941060941 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:03.258581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:03.267572 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d718a-299a-4408-91dd-18c941060941 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:03.301510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7f5c3-406d-4edc-b4b4-c4dd29b7e3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:03.302185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7f5c3-406d-4edc-b4b4-c4dd29b7e3c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:03.302461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7f5c3-406d-4edc-b4b4-c4dd29b7e3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:03.302731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7f5c3-406d-4edc-b4b4-c4dd29b7e3c4 None admin] PATH_INFO: `/v3/projects/134fe94171684b8d96893638e4870bcc/users/4b181d38588d44d4bb72fa319d2ba2f2/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:03.304600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7f5c3-406d-4edc-b4b4-c4dd29b7e3c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=134fe94171684b8d96893638e4870bcc, user_id=4b181d38588d44d4bb72fa319d2ba2f2, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:03.306134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7f5c3-406d-4edc-b4b4-c4dd29b7e3c4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:03.319131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 634/1273] 10.222.0.100 () {58 vars in 1444 bytes} [Thu Sep 1 17:27:03 2022] PUT /identity/v3/projects/134fe94171684b8d96893638e4870bcc/users/4b181d38588d44d4bb72fa319d2ba2f2/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:27:05.226927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae91af5-3d9d-49dd-bcf7-31bdc9b36f96 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:05.227459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae91af5-3d9d-49dd-bcf7-31bdc9b36f96 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:05.227459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae91af5-3d9d-49dd-bcf7-31bdc9b36f96 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:05.228818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 640/1274] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:27:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:27:05.233892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1732-2342-4d0b-b019-5d48258bf896 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:05.234307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1732-2342-4d0b-b019-5d48258bf896 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:05.234443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1732-2342-4d0b-b019-5d48258bf896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:05.255441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4deb1732-2342-4d0b-b019-5d48258bf896 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:05.256618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4deb1732-2342-4d0b-b019-5d48258bf896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:05.324324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 635/1275] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:27:05 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:27:05.593049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae91af5-3d9d-49dd-bcf7-31bdc9b36f96 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:05.696362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae91af5-3d9d-49dd-bcf7-31bdc9b36f96 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:05.713382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:05.732302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8027d69-ccaa-42b8-b784-70d419d3293f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:05.732926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8027d69-ccaa-42b8-b784-70d419d3293f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:05.733301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8027d69-ccaa-42b8-b784-70d419d3293f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:05.733681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8027d69-ccaa-42b8-b784-70d419d3293f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:05.747858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8027d69-ccaa-42b8-b784-70d419d3293f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d8027d69-ccaa-42b8-b784-70d419d3293f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:27:05.753165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:05.754318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:05.755242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 641/1276] 10.222.0.100 () {56 vars in 1101 bytes} [Thu Sep 1 17:27:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:05.760528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1732-2342-4d0b-b019-5d48258bf896 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:05.844199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:05.863940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1732-2342-4d0b-b019-5d48258bf896 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:05.896914 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6538a-9af3-4d9e-8863-5f9adf93bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:05.897507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6538a-9af3-4d9e-8863-5f9adf93bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:05.897814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6538a-9af3-4d9e-8863-5f9adf93bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:05.898086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6538a-9af3-4d9e-8863-5f9adf93bfb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:05.898772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6538a-9af3-4d9e-8863-5f9adf93bfb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:05.899073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6538a-9af3-4d9e-8863-5f9adf93bfb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:05.900427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6538a-9af3-4d9e-8863-5f9adf93bfb4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:05.909381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 636/1277] 10.222.0.100 () {56 vars in 1104 bytes} [Thu Sep 1 17:27:05 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:05.915842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8027d69-ccaa-42b8-b784-70d419d3293f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:06.025334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8027d69-ccaa-42b8-b784-70d419d3293f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:06.059330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663eb2b-8b45-451e-8f48-8b6cac518678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:06.059942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663eb2b-8b45-451e-8f48-8b6cac518678 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:06.060217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663eb2b-8b45-451e-8f48-8b6cac518678 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:06.060483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663eb2b-8b45-451e-8f48-8b6cac518678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:06.070184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663eb2b-8b45-451e-8f48-8b6cac518678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:06.071601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663eb2b-8b45-451e-8f48-8b6cac518678 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f663eb2b-8b45-451e-8f48-8b6cac518678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:27:06.086108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:06.087234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:27:06.088205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:27:06.088205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:27:06.088205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:06.088205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 642/1278] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:06.093389 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6538a-9af3-4d9e-8863-5f9adf93bfb4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:06.192194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6538a-9af3-4d9e-8863-5f9adf93bfb4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:06.226167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1db50-351f-4de6-8206-e2df9e0aa866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:06.226759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1db50-351f-4de6-8206-e2df9e0aa866 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:06.227034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1db50-351f-4de6-8206-e2df9e0aa866 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:06.227306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1db50-351f-4de6-8206-e2df9e0aa866 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:06.227932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1db50-351f-4de6-8206-e2df9e0aa866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:06.228236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1db50-351f-4de6-8206-e2df9e0aa866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:06.229610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1db50-351f-4de6-8206-e2df9e0aa866 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:06.241037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 637/1279] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:27:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:06.246781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663eb2b-8b45-451e-8f48-8b6cac518678 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:06.284478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:06.354312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663eb2b-8b45-451e-8f48-8b6cac518678 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:06.387272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c407da3-bfc6-4a1a-b708-cea84ca63a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:06.387889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c407da3-bfc6-4a1a-b708-cea84ca63a0c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:06.388169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c407da3-bfc6-4a1a-b708-cea84ca63a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:06.388440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c407da3-bfc6-4a1a-b708-cea84ca63a0c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:06.396812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c407da3-bfc6-4a1a-b708-cea84ca63a0c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:06.398284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c407da3-bfc6-4a1a-b708-cea84ca63a0c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8c407da3-bfc6-4a1a-b708-cea84ca63a0c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:27:06.409562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:06.410724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:06.411875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 643/1280] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:27:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:06.416543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1db50-351f-4de6-8206-e2df9e0aa866 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:06.420280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:06.513442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1db50-351f-4de6-8206-e2df9e0aa866 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:06.547248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d285a-2444-4f33-81e8-d72b2c346b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:06.547836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d285a-2444-4f33-81e8-d72b2c346b1f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:06.548115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d285a-2444-4f33-81e8-d72b2c346b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:06.548386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d285a-2444-4f33-81e8-d72b2c346b1f None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:06.549098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d285a-2444-4f33-81e8-d72b2c346b1f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:06.549449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d285a-2444-4f33-81e8-d72b2c346b1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:06.550895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d285a-2444-4f33-81e8-d72b2c346b1f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:06.567975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 638/1281] 10.222.0.100 () {56 vars in 1148 bytes} [Thu Sep 1 17:27:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:06.573756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c407da3-bfc6-4a1a-b708-cea84ca63a0c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:06.666323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:06.677698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c407da3-bfc6-4a1a-b708-cea84ca63a0c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:06.711802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6db9c-cb82-4acc-80a3-a66393c03a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:06.712408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6db9c-cb82-4acc-80a3-a66393c03a2e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:06.712690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6db9c-cb82-4acc-80a3-a66393c03a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:06.712962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6db9c-cb82-4acc-80a3-a66393c03a2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:06.779053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6db9c-cb82-4acc-80a3-a66393c03a2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:06.780427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6db9c-cb82-4acc-80a3-a66393c03a2e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:06.867271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:06.878721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 644/1282] 10.222.0.100 () {58 vars in 1287 bytes} [Thu Sep 1 17:27:06 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:27:06.884502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d285a-2444-4f33-81e8-d72b2c346b1f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:06.946198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:06.985963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d285a-2444-4f33-81e8-d72b2c346b1f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:07.023194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc56c-d46c-4cb3-8d4f-44b53979da3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:07.023822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc56c-d46c-4cb3-8d4f-44b53979da3f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:07.024102 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc56c-d46c-4cb3-8d4f-44b53979da3f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:07.024378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc56c-d46c-4cb3-8d4f-44b53979da3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:07.035222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bc56c-d46c-4cb3-8d4f-44b53979da3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:07.036599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bc56c-d46c-4cb3-8d4f-44b53979da3f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f96bc56c-d46c-4cb3-8d4f-44b53979da3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:27:07.051673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:07.052789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:27:07.054085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:27:07.054085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:27:07.054085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:27:07.054085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 639/1283] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:07.058541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6db9c-cb82-4acc-80a3-a66393c03a2e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:07.155229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6db9c-cb82-4acc-80a3-a66393c03a2e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:07.189234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620b1eb-c55a-444b-beca-70a97fb9ab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:07.189864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620b1eb-c55a-444b-beca-70a97fb9ab73 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:07.190169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620b1eb-c55a-444b-beca-70a97fb9ab73 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:07.190443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620b1eb-c55a-444b-beca-70a97fb9ab73 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:07.191084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620b1eb-c55a-444b-beca-70a97fb9ab73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:07.191385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620b1eb-c55a-444b-beca-70a97fb9ab73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:07.192736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620b1eb-c55a-444b-beca-70a97fb9ab73 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:07.203612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 645/1284] 10.222.0.100 () {56 vars in 1114 bytes} [Thu Sep 1 17:27:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:07.209681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc56c-d46c-4cb3-8d4f-44b53979da3f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:07.306906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc56c-d46c-4cb3-8d4f-44b53979da3f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:07.322754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:07.345405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7dd55-95b6-4168-a84e-a1b495c37a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:07.346054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7dd55-95b6-4168-a84e-a1b495c37a02 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:07.346886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7dd55-95b6-4168-a84e-a1b495c37a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:07.347159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7dd55-95b6-4168-a84e-a1b495c37a02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:07.353587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7dd55-95b6-4168-a84e-a1b495c37a02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:07.355031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7dd55-95b6-4168-a84e-a1b495c37a02 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-18d7dd55-95b6-4168-a84e-a1b495c37a02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:27:07.366586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:07.367621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:07.368979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:27:07.368979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:27:07.368979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:27:07.368979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:27:07.368979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:27:07.368979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:27:07.368979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:27:07.368979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Sep 01 17:27:07.368979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 640/1285] 10.222.0.100 () {56 vars in 1116 bytes} [Thu Sep 1 17:27:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:07.373565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620b1eb-c55a-444b-beca-70a97fb9ab73 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:07.422871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:07.470724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620b1eb-c55a-444b-beca-70a97fb9ab73 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:07.502900 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb18efe-6829-4a53-a122-356cf59fccae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:07.503292 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb18efe-6829-4a53-a122-356cf59fccae None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:07.503566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb18efe-6829-4a53-a122-356cf59fccae None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:07.503833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb18efe-6829-4a53-a122-356cf59fccae None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:07.504508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb18efe-6829-4a53-a122-356cf59fccae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:07.504807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb18efe-6829-4a53-a122-356cf59fccae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:07.506249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb18efe-6829-4a53-a122-356cf59fccae None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:07.518945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 646/1286] 10.222.0.100 () {56 vars in 1153 bytes} [Thu Sep 1 17:27:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:07.525158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7dd55-95b6-4168-a84e-a1b495c37a02 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:07.624848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7dd55-95b6-4168-a84e-a1b495c37a02 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:07.659898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad3529-b93b-41db-8a3f-36736527b575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:07.660580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad3529-b93b-41db-8a3f-36736527b575 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:07.660856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad3529-b93b-41db-8a3f-36736527b575 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:07.661127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad3529-b93b-41db-8a3f-36736527b575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:07.661880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad3529-b93b-41db-8a3f-36736527b575 None admin] RBAC: Adding query filter params (role.id=8857e6335b964c8ba99f8b0499179c75, scope.project.id=134fe94171684b8d96893638e4870bcc, user.id=4b181d38588d44d4bb72fa319d2ba2f2) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:07.662185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad3529-b93b-41db-8a3f-36736527b575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:07.663602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad3529-b93b-41db-8a3f-36736527b575 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:07.669429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 641/1287] 10.222.0.100 () {56 vars in 1379 bytes} [Thu Sep 1 17:27:07 2022] GET /identity/v3/role_assignments?user.id=4b181d38588d44d4bb72fa319d2ba2f2&scope.project.id=134fe94171684b8d96893638e4870bcc&role.id=8857e6335b964c8ba99f8b0499179c75 => generated 610 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:09.585713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e92bb-661f-4488-8be4-3ae61bfc80dc None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:09.586254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e92bb-661f-4488-8be4-3ae61bfc80dc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:09.586254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e92bb-661f-4488-8be4-3ae61bfc80dc None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:09.587592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 647/1288] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:27:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:27:09.592763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793c34-2879-4385-a1ea-60ce3fa458d4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:09.593168 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793c34-2879-4385-a1ea-60ce3fa458d4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:09.593310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793c34-2879-4385-a1ea-60ce3fa458d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:09.611712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-af793c34-2879-4385-a1ea-60ce3fa458d4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:09.613550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-af793c34-2879-4385-a1ea-60ce3fa458d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:09.643164 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:09.682075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 642/1289] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:27:09 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:27:09.951313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e92bb-661f-4488-8be4-3ae61bfc80dc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:10.063084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e92bb-661f-4488-8be4-3ae61bfc80dc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:10.074235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:10.097758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0e1dd-161f-4d40-bdaf-c26f066a4212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:10.098377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0e1dd-161f-4d40-bdaf-c26f066a4212 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:10.098658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0e1dd-161f-4d40-bdaf-c26f066a4212 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:10.098933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0e1dd-161f-4d40-bdaf-c26f066a4212 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:10.099512 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0e1dd-161f-4d40-bdaf-c26f066a4212 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:10.100863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0e1dd-161f-4d40-bdaf-c26f066a4212 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-bdc0e1dd-161f-4d40-bdaf-c26f066a4212 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:10.109688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:10.111041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:10.112886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 648/1290] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:27:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:10.117335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793c34-2879-4385-a1ea-60ce3fa458d4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:10.213371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793c34-2879-4385-a1ea-60ce3fa458d4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:10.246828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c4350-b19f-45e7-b193-85dddf1e64c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:10.247422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c4350-b19f-45e7-b193-85dddf1e64c3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:10.247702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c4350-b19f-45e7-b193-85dddf1e64c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:10.247975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c4350-b19f-45e7-b193-85dddf1e64c3 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:10.248598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c4350-b19f-45e7-b193-85dddf1e64c3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:10.248900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c4350-b19f-45e7-b193-85dddf1e64c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:10.250299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c4350-b19f-45e7-b193-85dddf1e64c3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:10.255024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 643/1291] 10.222.0.100 () {56 vars in 1121 bytes} [Thu Sep 1 17:27:10 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:10.260793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0e1dd-161f-4d40-bdaf-c26f066a4212 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:10.358127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0e1dd-161f-4d40-bdaf-c26f066a4212 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:10.390677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd3c0e-3012-4aeb-97a6-a8e23226668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:10.391267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd3c0e-3012-4aeb-97a6-a8e23226668e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:10.391547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd3c0e-3012-4aeb-97a6-a8e23226668e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:10.391818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd3c0e-3012-4aeb-97a6-a8e23226668e None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:10.392468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd3c0e-3012-4aeb-97a6-a8e23226668e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:10.392765 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd3c0e-3012-4aeb-97a6-a8e23226668e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:10.394733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd3c0e-3012-4aeb-97a6-a8e23226668e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:10.399026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 649/1292] 10.222.0.100 () {56 vars in 1121 bytes} [Thu Sep 1 17:27:10 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:12.309270 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45140261-b5c7-413c-b4a4-0e75009d1392 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:12.309869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45140261-b5c7-413c-b4a4-0e75009d1392 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:12.309869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45140261-b5c7-413c-b4a4-0e75009d1392 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:12.311188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 644/1293] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:27:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:27:12.316772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901844e0-64c3-4353-8c96-32fd2d4e297b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:12.317192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901844e0-64c3-4353-8c96-32fd2d4e297b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:12.317328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901844e0-64c3-4353-8c96-32fd2d4e297b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:12.335412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-901844e0-64c3-4353-8c96-32fd2d4e297b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:12.336592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-901844e0-64c3-4353-8c96-32fd2d4e297b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:12.366488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:12.402366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 650/1294] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:27:12 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:27:12.675408 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45140261-b5c7-413c-b4a4-0e75009d1392 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:12.708444 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:12.789902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45140261-b5c7-413c-b4a4-0e75009d1392 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:12.822876 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8c3ff-e4e3-43c5-8e29-9e5873944cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:12.823826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8c3ff-e4e3-43c5-8e29-9e5873944cf6 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:12.824106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8c3ff-e4e3-43c5-8e29-9e5873944cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:12.824487 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8c3ff-e4e3-43c5-8e29-9e5873944cf6 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:12.825163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8c3ff-e4e3-43c5-8e29-9e5873944cf6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:12.826665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8c3ff-e4e3-43c5-8e29-9e5873944cf6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:12.842658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 645/1295] 10.222.0.100 () {60 vars in 1146 bytes} [Thu Sep 1 17:27:12 2022] POST /identity/v3/services => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:27:14.768534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbae18-d37e-4f3d-bfa7-1240e2f9f903 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:14.769132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbae18-d37e-4f3d-bfa7-1240e2f9f903 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:14.769132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbae18-d37e-4f3d-bfa7-1240e2f9f903 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:14.770450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 651/1296] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:27:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:27:14.775828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cbeb-6168-4ee3-8879-f00c347a7a75 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:14.776280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cbeb-6168-4ee3-8879-f00c347a7a75 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:14.776380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cbeb-6168-4ee3-8879-f00c347a7a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:14.795094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2b17cbeb-6168-4ee3-8879-f00c347a7a75 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:14.796275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2b17cbeb-6168-4ee3-8879-f00c347a7a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:14.868575 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 646/1297] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:27:14 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:27:15.162385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbae18-d37e-4f3d-bfa7-1240e2f9f903 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:15.263156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbae18-d37e-4f3d-bfa7-1240e2f9f903 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:15.296629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d37960-74d0-40a8-a89a-e683e2e3a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:15.297239 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d37960-74d0-40a8-a89a-e683e2e3a1bd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:15.297516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d37960-74d0-40a8-a89a-e683e2e3a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:15.297811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d37960-74d0-40a8-a89a-e683e2e3a1bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:15.298383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d37960-74d0-40a8-a89a-e683e2e3a1bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:15.300417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d37960-74d0-40a8-a89a-e683e2e3a1bd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f2d37960-74d0-40a8-a89a-e683e2e3a1bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:15.308991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:15.310307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:15.311495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 652/1298] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:27:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:15.316332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17cbeb-6168-4ee3-8879-f00c347a7a75 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:15.334499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:15.414452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17cbeb-6168-4ee3-8879-f00c347a7a75 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:15.447438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed07d45-69df-47c3-8fb3-9afb9224ec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:15.448067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed07d45-69df-47c3-8fb3-9afb9224ec3b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:15.448353 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed07d45-69df-47c3-8fb3-9afb9224ec3b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:15.448624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed07d45-69df-47c3-8fb3-9afb9224ec3b None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:15.449245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed07d45-69df-47c3-8fb3-9afb9224ec3b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:15.449543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed07d45-69df-47c3-8fb3-9afb9224ec3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:15.450927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed07d45-69df-47c3-8fb3-9afb9224ec3b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:15.455973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 647/1299] 10.222.0.100 () {56 vars in 1121 bytes} [Thu Sep 1 17:27:15 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:15.462092 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d37960-74d0-40a8-a89a-e683e2e3a1bd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:15.494985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:15.564460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d37960-74d0-40a8-a89a-e683e2e3a1bd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:15.599445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b062efe-96f7-4466-b3d3-52411a76ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:15.600039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b062efe-96f7-4466-b3d3-52411a76ce9c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:15.600318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b062efe-96f7-4466-b3d3-52411a76ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:15.600586 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b062efe-96f7-4466-b3d3-52411a76ce9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:15.601279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b062efe-96f7-4466-b3d3-52411a76ce9c None admin] RBAC: Adding query filter params (interface=public, service_id=656a50f8e1a24799b6d63f8322636dcf, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:15.601608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b062efe-96f7-4466-b3d3-52411a76ce9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:15.602969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b062efe-96f7-4466-b3d3-52411a76ce9c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:15.608164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 653/1300] 10.222.0.100 () {56 vars in 1256 bytes} [Thu Sep 1 17:27:15 2022] GET /identity/v3/endpoints?service_id=656a50f8e1a24799b6d63f8322636dcf&interface=public®ion_id=RegionOne => generated 195 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:15.614957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed07d45-69df-47c3-8fb3-9afb9224ec3b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:15.695273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:15.713537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed07d45-69df-47c3-8fb3-9afb9224ec3b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:15.747555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff49665-998d-48b2-bc6a-8992428f076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:15.748146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff49665-998d-48b2-bc6a-8992428f076a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:15.748421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff49665-998d-48b2-bc6a-8992428f076a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:15.748689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff49665-998d-48b2-bc6a-8992428f076a None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:15.749276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff49665-998d-48b2-bc6a-8992428f076a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:15.750650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff49665-998d-48b2-bc6a-8992428f076a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:15.757185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 648/1301] 10.222.0.100 () {56 vars in 1092 bytes} [Thu Sep 1 17:27:15 2022] GET /identity/v3/services => generated 2201 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:27:17.655948 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2719a-c8fe-40e8-bdde-163da89e9f8c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:17.656642 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2719a-c8fe-40e8-bdde-163da89e9f8c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:17.656742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2719a-c8fe-40e8-bdde-163da89e9f8c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:17.658149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 654/1302] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:27:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:27:17.664578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51ff2e-37c5-45cf-8203-e5b01d329c4d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:17.665023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51ff2e-37c5-45cf-8203-e5b01d329c4d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:17.665123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51ff2e-37c5-45cf-8203-e5b01d329c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:17.683795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0c51ff2e-37c5-45cf-8203-e5b01d329c4d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:27:17.684982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0c51ff2e-37c5-45cf-8203-e5b01d329c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:27:17.749802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 649/1303] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:27:17 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:27:18.019231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2719a-c8fe-40e8-bdde-163da89e9f8c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:18.113509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:18.120441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2719a-c8fe-40e8-bdde-163da89e9f8c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:18.153244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663c11d-4791-40b9-b8a6-0570408a0282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:18.153891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663c11d-4791-40b9-b8a6-0570408a0282 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:18.154175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663c11d-4791-40b9-b8a6-0570408a0282 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:18.154450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663c11d-4791-40b9-b8a6-0570408a0282 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:18.155027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663c11d-4791-40b9-b8a6-0570408a0282 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:18.156630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663c11d-4791-40b9-b8a6-0570408a0282 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d663c11d-4791-40b9-b8a6-0570408a0282 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:27:18.165855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:27:18.167029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Sep 01 17:27:18.168064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 655/1304] 10.222.0.100 () {56 vars in 1122 bytes} [Thu Sep 1 17:27:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:27:18.173081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51ff2e-37c5-45cf-8203-e5b01d329c4d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:18.246635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:18.270276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51ff2e-37c5-45cf-8203-e5b01d329c4d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:18.302695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac2456-6fde-4880-a167-3d0969843f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:18.303296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2456-6fde-4880-a167-3d0969843f5f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:18.303565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2456-6fde-4880-a167-3d0969843f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:18.303832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2456-6fde-4880-a167-3d0969843f5f None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:18.304448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac2456-6fde-4880-a167-3d0969843f5f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:27:18.304745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac2456-6fde-4880-a167-3d0969843f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:18.306194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac2456-6fde-4880-a167-3d0969843f5f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:18.311098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 650/1305] 10.222.0.100 () {56 vars in 1121 bytes} [Thu Sep 1 17:27:18 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:27:18.317839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663c11d-4791-40b9-b8a6-0570408a0282 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:27:18.417945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663c11d-4791-40b9-b8a6-0570408a0282 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:27:18.451816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eda842-c359-4432-b53f-ae57f3dd456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:27:18.452409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eda842-c359-4432-b53f-ae57f3dd456c None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:27:18.452695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eda842-c359-4432-b53f-ae57f3dd456c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:27:18.452972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eda842-c359-4432-b53f-ae57f3dd456c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:27:18.453649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eda842-c359-4432-b53f-ae57f3dd456c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:27:18.455009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eda842-c359-4432-b53f-ae57f3dd456c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:27:18.474373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 656/1306] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:27:18 2022] POST /identity/v3/endpoints => generated 333 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:27:28.308157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=30 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:27:28.470451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:24.182786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7614b83-4c18-4fdd-a5e1-b46d56eaf9b5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:24.183373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7614b83-4c18-4fdd-a5e1-b46d56eaf9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:24.183661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7614b83-4c18-4fdd-a5e1-b46d56eaf9b5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:24.185463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 651/1307] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:24.191474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06092d23-7d15-4c0f-8f00-6a6855142bb3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:24.192067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06092d23-7d15-4c0f-8f00-6a6855142bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:24.192638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06092d23-7d15-4c0f-8f00-6a6855142bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:24.288522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-06092d23-7d15-4c0f-8f00-6a6855142bb3 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:24.289954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-06092d23-7d15-4c0f-8f00-6a6855142bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:24.434031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 657/1308] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:24.746368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3f60-810f-45a2-9839-49fa2b6d889d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:24.752527 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3f60-810f-45a2-9839-49fa2b6d889d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:24.753145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3f60-810f-45a2-9839-49fa2b6d889d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:24.754767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 652/1309] 10.222.0.100 () {54 vars in 970 bytes} [Thu Sep 1 17:28:24 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:24.769752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c40257-7b98-4dab-8ff8-8da69832b0e0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:24.770231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c40257-7b98-4dab-8ff8-8da69832b0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:24.770962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c40257-7b98-4dab-8ff8-8da69832b0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:24.866650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-27c40257-7b98-4dab-8ff8-8da69832b0e0 None None] MFA Rules not processed for user `bbe180b45f2e49e2abbf6e833ce0d44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:24.868018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-27c40257-7b98-4dab-8ff8-8da69832b0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:24.890115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:24.950741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 658/1310] 10.222.0.100 () {58 vars in 1067 bytes} [Thu Sep 1 17:28:24 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:24.960143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3f60-810f-45a2-9839-49fa2b6d889d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:24.962055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1b6c3f60-810f-45a2-9839-49fa2b6d889d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:25.084933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3f60-810f-45a2-9839-49fa2b6d889d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:25.179525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ba510-bec5-4b3a-837e-95ddf95a70e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:25.180061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ba510-bec5-4b3a-837e-95ddf95a70e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:25.180338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ba510-bec5-4b3a-837e-95ddf95a70e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:25.180604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ba510-bec5-4b3a-837e-95ddf95a70e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:25.182322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-de1ba510-bec5-4b3a-837e-95ddf95a70e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:25.248785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:25.315910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ba510-bec5-4b3a-837e-95ddf95a70e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:25.318236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ba510-bec5-4b3a-837e-95ddf95a70e3 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:25.345435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 653/1311] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:27.721793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd9a96-de9b-401e-8106-237c5c371974 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:27.722393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd9a96-de9b-401e-8106-237c5c371974 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:27.722836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd9a96-de9b-401e-8106-237c5c371974 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:27.725305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 659/1312] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:27 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:27.732950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33a73e-56ca-4573-a883-e2ce371fa716 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:27.733540 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33a73e-56ca-4573-a883-e2ce371fa716 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:27.733540 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33a73e-56ca-4573-a883-e2ce371fa716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:27.822859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5f33a73e-56ca-4573-a883-e2ce371fa716 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:27.824987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5f33a73e-56ca-4573-a883-e2ce371fa716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:27.971116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 654/1313] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:28.262258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd9a96-de9b-401e-8106-237c5c371974 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:28.263800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f5dd9a96-de9b-401e-8106-237c5c371974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:28.393386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd9a96-de9b-401e-8106-237c5c371974 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:28.468715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:28.472550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff073ee-c9a9-452e-a860-fd8e709cbe4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:28.473442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff073ee-c9a9-452e-a860-fd8e709cbe4e service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:28.473864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff073ee-c9a9-452e-a860-fd8e709cbe4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:28.474212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff073ee-c9a9-452e-a860-fd8e709cbe4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:28.476380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6ff073ee-c9a9-452e-a860-fd8e709cbe4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:28.592631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff073ee-c9a9-452e-a860-fd8e709cbe4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:28.594053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff073ee-c9a9-452e-a860-fd8e709cbe4e service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:28.615037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 660/1314] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:30.877105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf35d89-6b40-4655-9cfe-9fccd63932e2 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:30.877694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf35d89-6b40-4655-9cfe-9fccd63932e2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:30.877694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf35d89-6b40-4655-9cfe-9fccd63932e2 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:30.879023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 655/1315] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:30.884074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5262-dbf1-45fd-8b4b-f9e78c72b1e2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:30.884457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5262-dbf1-45fd-8b4b-f9e78c72b1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:30.884621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5262-dbf1-45fd-8b4b-f9e78c72b1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:30.949512 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6dff5262-dbf1-45fd-8b4b-f9e78c72b1e2 None None] MFA Rules not processed for user `400981e92abc4ccea32372bcc765cfa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:30.950673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6dff5262-dbf1-45fd-8b4b-f9e78c72b1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:31.007243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:31.052713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 661/1316] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:30 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:31.351425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda22cf-a4cb-4524-8a49-24a646531ac7 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:31.351708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda22cf-a4cb-4524-8a49-24a646531ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:31.351999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda22cf-a4cb-4524-8a49-24a646531ac7 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:31.353344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 656/1317] 10.222.0.100 () {54 vars in 970 bytes} [Thu Sep 1 17:28:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:31.359344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be7bd5-24ad-4a4c-82ed-a24d548e19c9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:31.359655 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be7bd5-24ad-4a4c-82ed-a24d548e19c9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:31.360428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be7bd5-24ad-4a4c-82ed-a24d548e19c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:31.435636 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-76be7bd5-24ad-4a4c-82ed-a24d548e19c9 None None] MFA Rules not processed for user `bbe180b45f2e49e2abbf6e833ce0d44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:31.437009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-76be7bd5-24ad-4a4c-82ed-a24d548e19c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:31.511987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 662/1318] 10.222.0.100 () {58 vars in 1067 bytes} [Thu Sep 1 17:28:31 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:31.521339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda22cf-a4cb-4524-8a49-24a646531ac7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:31.522955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2cda22cf-a4cb-4524-8a49-24a646531ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:31.533039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:31.626803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda22cf-a4cb-4524-8a49-24a646531ac7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:31.700409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f197a-5cb6-4dfb-988f-8a94158c9a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:31.701013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f197a-5cb6-4dfb-988f-8a94158c9a93 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:31.701295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f197a-5cb6-4dfb-988f-8a94158c9a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:31.701593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f197a-5cb6-4dfb-988f-8a94158c9a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:31.703205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aa0f197a-5cb6-4dfb-988f-8a94158c9a93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:31.768322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:31.816769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f197a-5cb6-4dfb-988f-8a94158c9a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:31.818240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f197a-5cb6-4dfb-988f-8a94158c9a93 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:31.836522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 657/1319] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:31 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:28:34.722951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a433de-2c28-45f0-940c-df445d52e204 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:34.723513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a433de-2c28-45f0-940c-df445d52e204 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:34.723513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a433de-2c28-45f0-940c-df445d52e204 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:34.724828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 663/1320] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:34.729858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968a754-f966-4916-9093-910d85e824af None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:34.730283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968a754-f966-4916-9093-910d85e824af None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:34.730407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968a754-f966-4916-9093-910d85e824af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:34.798396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5968a754-f966-4916-9093-910d85e824af None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:34.799629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5968a754-f966-4916-9093-910d85e824af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:34.894120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 658/1321] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:35.179156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a433de-2c28-45f0-940c-df445d52e204 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:35.180694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-14a433de-2c28-45f0-940c-df445d52e204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:35.231794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:35.286102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a433de-2c28-45f0-940c-df445d52e204 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:35.358698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9168d61-dabe-4243-9b04-503f13f2ed05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:35.359291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9168d61-dabe-4243-9b04-503f13f2ed05 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:35.359565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9168d61-dabe-4243-9b04-503f13f2ed05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:35.360432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9168d61-dabe-4243-9b04-503f13f2ed05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:35.362056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a9168d61-dabe-4243-9b04-503f13f2ed05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:35.457118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:35.471128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9168d61-dabe-4243-9b04-503f13f2ed05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:35.472558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9168d61-dabe-4243-9b04-503f13f2ed05 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:35.490280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 664/1322] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:38.154109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a85764-f657-4d9d-93a7-9be56092fc72 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:38.154734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a85764-f657-4d9d-93a7-9be56092fc72 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:38.154734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a85764-f657-4d9d-93a7-9be56092fc72 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:38.156025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 659/1323] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:38.162617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e0511-97f8-4a8e-867b-60292170e624 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:38.163309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e0511-97f8-4a8e-867b-60292170e624 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:38.163545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e0511-97f8-4a8e-867b-60292170e624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:38.230097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b23e0511-97f8-4a8e-867b-60292170e624 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:38.231300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b23e0511-97f8-4a8e-867b-60292170e624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:38.330155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 665/1324] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:38.615571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a85764-f657-4d9d-93a7-9be56092fc72 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:38.617169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a9a85764-f657-4d9d-93a7-9be56092fc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:38.636710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:38.725681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a85764-f657-4d9d-93a7-9be56092fc72 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:38.800778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec545ff-c57d-4f4f-bf28-0f20917d0123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:38.801282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec545ff-c57d-4f4f-bf28-0f20917d0123 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:38.801562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec545ff-c57d-4f4f-bf28-0f20917d0123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:38.801860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec545ff-c57d-4f4f-bf28-0f20917d0123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:38.803469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6ec545ff-c57d-4f4f-bf28-0f20917d0123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:38.874183 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:38.919277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec545ff-c57d-4f4f-bf28-0f20917d0123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:38.920721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec545ff-c57d-4f4f-bf28-0f20917d0123 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:38.938799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 660/1325] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:40.928482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818527c-8872-4e1f-9cab-98c71f00ca3f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:40.929109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818527c-8872-4e1f-9cab-98c71f00ca3f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:40.929109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818527c-8872-4e1f-9cab-98c71f00ca3f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:40.930623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 666/1326] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:40.935921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d4cd-f104-4629-8b8b-dae6f599b460 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:40.936413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d4cd-f104-4629-8b8b-dae6f599b460 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:40.936497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d4cd-f104-4629-8b8b-dae6f599b460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:41.009406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dc72d4cd-f104-4629-8b8b-dae6f599b460 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:41.010664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dc72d4cd-f104-4629-8b8b-dae6f599b460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:41.108542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 661/1327] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:41.390322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818527c-8872-4e1f-9cab-98c71f00ca3f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:41.392687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e818527c-8872-4e1f-9cab-98c71f00ca3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:41.455088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:41.511937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818527c-8872-4e1f-9cab-98c71f00ca3f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:41.592719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3f8a5-09d2-40ae-9437-cfac19486c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:41.593328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3f8a5-09d2-40ae-9437-cfac19486c41 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:41.593634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3f8a5-09d2-40ae-9437-cfac19486c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:41.593913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3f8a5-09d2-40ae-9437-cfac19486c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:41.595642 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c1d3f8a5-09d2-40ae-9437-cfac19486c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:41.706716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:41.708799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3f8a5-09d2-40ae-9437-cfac19486c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:41.710283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3f8a5-09d2-40ae-9437-cfac19486c41 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:41.729182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 667/1328] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:43.728697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80d29a-94ff-4f03-a6c2-8666b989d4b3 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:43.729288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80d29a-94ff-4f03-a6c2-8666b989d4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:43.729288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80d29a-94ff-4f03-a6c2-8666b989d4b3 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:43.730669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 662/1329] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:43.737216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b4eeb-9401-4efb-8598-7fa6d4823037 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:43.737806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b4eeb-9401-4efb-8598-7fa6d4823037 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:43.737806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b4eeb-9401-4efb-8598-7fa6d4823037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:43.806804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-086b4eeb-9401-4efb-8598-7fa6d4823037 None None] MFA Rules not processed for user `400981e92abc4ccea32372bcc765cfa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:43.808035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-086b4eeb-9401-4efb-8598-7fa6d4823037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:43.912469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 668/1330] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:43 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:44.197373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80d29a-94ff-4f03-a6c2-8666b989d4b3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:44.198933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3d80d29a-94ff-4f03-a6c2-8666b989d4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:44.232240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:44.310031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80d29a-94ff-4f03-a6c2-8666b989d4b3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:44.395061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346b8b5-6f8e-41c5-aeea-cb6fb7661e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:44.396019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346b8b5-6f8e-41c5-aeea-cb6fb7661e19 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:44.396019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346b8b5-6f8e-41c5-aeea-cb6fb7661e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:44.396209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346b8b5-6f8e-41c5-aeea-cb6fb7661e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:44.397880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b346b8b5-6f8e-41c5-aeea-cb6fb7661e19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:44.477675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:44.516535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346b8b5-6f8e-41c5-aeea-cb6fb7661e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:44.518022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346b8b5-6f8e-41c5-aeea-cb6fb7661e19 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:44.536993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 663/1331] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:44 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:46.809041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6d5cd-ad42-4f89-b074-4d4a6eaa3ef6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:46.809672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6d5cd-ad42-4f89-b074-4d4a6eaa3ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:46.809672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6d5cd-ad42-4f89-b074-4d4a6eaa3ef6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:46.810985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 669/1332] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:46.818341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a87b1-4217-4e90-8e04-31ab4c10215a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:46.818842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a87b1-4217-4e90-8e04-31ab4c10215a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:46.818944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a87b1-4217-4e90-8e04-31ab4c10215a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:46.901030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-725a87b1-4217-4e90-8e04-31ab4c10215a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:46.902556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-725a87b1-4217-4e90-8e04-31ab4c10215a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:47.010232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 664/1333] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:47.311526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6d5cd-ad42-4f89-b074-4d4a6eaa3ef6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:47.315193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6df6d5cd-ad42-4f89-b074-4d4a6eaa3ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:47.403578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:47.439299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6d5cd-ad42-4f89-b074-4d4a6eaa3ef6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:47.513165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9de1cc-471a-496a-a990-13bdea91b446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:47.513803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9de1cc-471a-496a-a990-13bdea91b446 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:47.514084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9de1cc-471a-496a-a990-13bdea91b446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:47.514358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9de1cc-471a-496a-a990-13bdea91b446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:47.516504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3e9de1cc-471a-496a-a990-13bdea91b446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:47.625396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9de1cc-471a-496a-a990-13bdea91b446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:47.626662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9de1cc-471a-496a-a990-13bdea91b446 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:47.637422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:47.645791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 670/1334] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:49.656751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63291f16-3de3-46b3-9678-613e8d56de47 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:49.657653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63291f16-3de3-46b3-9678-613e8d56de47 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:49.657653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63291f16-3de3-46b3-9678-613e8d56de47 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:49.658939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 665/1335] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:49.664376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454f3cc-9dc3-4078-9efb-524d3047f3c0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:49.664835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454f3cc-9dc3-4078-9efb-524d3047f3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:49.664931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454f3cc-9dc3-4078-9efb-524d3047f3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:49.730423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5454f3cc-9dc3-4078-9efb-524d3047f3c0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:49.731612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5454f3cc-9dc3-4078-9efb-524d3047f3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:49.828316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 671/1336] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:50.114239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63291f16-3de3-46b3-9678-613e8d56de47 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:50.115822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-63291f16-3de3-46b3-9678-613e8d56de47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:50.157900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:50.223688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63291f16-3de3-46b3-9678-613e8d56de47 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:50.299477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323bb46-78e9-45c5-9c8e-a2065d7bd8f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:50.300081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323bb46-78e9-45c5-9c8e-a2065d7bd8f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:50.300365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323bb46-78e9-45c5-9c8e-a2065d7bd8f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:50.300635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323bb46-78e9-45c5-9c8e-a2065d7bd8f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:50.302305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4323bb46-78e9-45c5-9c8e-a2065d7bd8f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:50.390055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:50.419150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323bb46-78e9-45c5-9c8e-a2065d7bd8f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:50.420595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323bb46-78e9-45c5-9c8e-a2065d7bd8f6 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:50.438772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 666/1337] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:52.721965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdce8c-7558-404f-a269-0926391f378d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:52.722525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdce8c-7558-404f-a269-0926391f378d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:52.722525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdce8c-7558-404f-a269-0926391f378d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:52.723874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 672/1338] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:52.729297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caffd1-fe82-4c4c-8d79-af61e5be620f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:52.729763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caffd1-fe82-4c4c-8d79-af61e5be620f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:52.729877 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caffd1-fe82-4c4c-8d79-af61e5be620f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:52.795548 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-25caffd1-fe82-4c4c-8d79-af61e5be620f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:52.797407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-25caffd1-fe82-4c4c-8d79-af61e5be620f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:52.902599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 667/1339] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:53.190979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdce8c-7558-404f-a269-0926391f378d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:53.192522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-14bdce8c-7558-404f-a269-0926391f378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:53.270598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:53.299515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdce8c-7558-404f-a269-0926391f378d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:53.372064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff31c51-afd9-4c41-82be-1c3da70b9782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:53.372666 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31c51-afd9-4c41-82be-1c3da70b9782 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:53.372946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31c51-afd9-4c41-82be-1c3da70b9782 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:53.373217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31c51-afd9-4c41-82be-1c3da70b9782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:53.374878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3ff31c51-afd9-4c41-82be-1c3da70b9782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:53.484735 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff31c51-afd9-4c41-82be-1c3da70b9782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:53.486375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff31c51-afd9-4c41-82be-1c3da70b9782 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:53.504850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 673/1340] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:28:57.383810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd222ec-cf85-49b6-a0a0-a8a28f0bf82d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:57.384286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd222ec-cf85-49b6-a0a0-a8a28f0bf82d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:57.384381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd222ec-cf85-49b6-a0a0-a8a28f0bf82d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:57.385724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 668/1341] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:28:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:28:57.390951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c366f84-a268-4b6b-a06f-696f015140e2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:57.391467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c366f84-a268-4b6b-a06f-696f015140e2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:57.391583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c366f84-a268-4b6b-a06f-696f015140e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:57.397073 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:57.462486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2c366f84-a268-4b6b-a06f-696f015140e2 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:28:57.463683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2c366f84-a268-4b6b-a06f-696f015140e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:57.561072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 674/1342] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:28:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:28:57.854996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd222ec-cf85-49b6-a0a0-a8a28f0bf82d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:28:57.856516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4dd222ec-cf85-49b6-a0a0-a8a28f0bf82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:57.905769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:57.962806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd222ec-cf85-49b6-a0a0-a8a28f0bf82d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:28:58.040875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe22d72-e892-4aca-bc8b-5b7f420e5bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:28:58.041472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe22d72-e892-4aca-bc8b-5b7f420e5bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:28:58.041777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe22d72-e892-4aca-bc8b-5b7f420e5bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:28:58.042052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe22d72-e892-4aca-bc8b-5b7f420e5bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:28:58.043655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-afe22d72-e892-4aca-bc8b-5b7f420e5bb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:28:58.137018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:28:58.154232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe22d72-e892-4aca-bc8b-5b7f420e5bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:28:58.155652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe22d72-e892-4aca-bc8b-5b7f420e5bb3 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:28:58.174057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 669/1343] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:28:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:00.818970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d8f05-f50d-4623-a6f7-4c88b93764c1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:00.819460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d8f05-f50d-4623-a6f7-4c88b93764c1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:00.819556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d8f05-f50d-4623-a6f7-4c88b93764c1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:00.820997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 675/1344] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:00.826277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acf054-b54d-4a7a-8d47-0cd17a8af3d9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:00.826734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acf054-b54d-4a7a-8d47-0cd17a8af3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:00.826824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acf054-b54d-4a7a-8d47-0cd17a8af3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:00.896833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d5acf054-b54d-4a7a-8d47-0cd17a8af3d9 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:00.898059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d5acf054-b54d-4a7a-8d47-0cd17a8af3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:01.008936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 670/1345] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:01.293978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d8f05-f50d-4623-a6f7-4c88b93764c1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:01.295601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-694d8f05-f50d-4623-a6f7-4c88b93764c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:01.385467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:01.403826 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d8f05-f50d-4623-a6f7-4c88b93764c1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:01.483983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792bf1e-9597-4427-956c-1e1a6b299ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:01.484758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792bf1e-9597-4427-956c-1e1a6b299ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:01.485122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792bf1e-9597-4427-956c-1e1a6b299ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:01.485395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792bf1e-9597-4427-956c-1e1a6b299ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:01.487176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9792bf1e-9597-4427-956c-1e1a6b299ab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:01.603361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792bf1e-9597-4427-956c-1e1a6b299ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:01.604694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792bf1e-9597-4427-956c-1e1a6b299ab4 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:01.623290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 676/1346] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:05.607298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9803f6-ee39-4819-92ab-a05539b4e414 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:05.607837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9803f6-ee39-4819-92ab-a05539b4e414 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:05.607837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9803f6-ee39-4819-92ab-a05539b4e414 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:05.610096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 671/1347] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:05 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:29:05.615811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26791ef9-8972-4fd9-a53b-e9399c53660c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:05.616556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26791ef9-8972-4fd9-a53b-e9399c53660c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:05.616556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26791ef9-8972-4fd9-a53b-e9399c53660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:05.634244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:05.697444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-26791ef9-8972-4fd9-a53b-e9399c53660c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:05.698747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-26791ef9-8972-4fd9-a53b-e9399c53660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:05.830287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 677/1348] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:29:06.117145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9803f6-ee39-4819-92ab-a05539b4e414 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:06.118686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-de9803f6-ee39-4819-92ab-a05539b4e414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:06.184023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:06.237457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9803f6-ee39-4819-92ab-a05539b4e414 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:06.325198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564c5b0-22b4-4f7c-8b20-b30b115f32db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:06.326043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564c5b0-22b4-4f7c-8b20-b30b115f32db service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:06.326388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564c5b0-22b4-4f7c-8b20-b30b115f32db service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:06.326820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564c5b0-22b4-4f7c-8b20-b30b115f32db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:06.329214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c564c5b0-22b4-4f7c-8b20-b30b115f32db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:06.458752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:06.463207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564c5b0-22b4-4f7c-8b20-b30b115f32db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:06.464737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564c5b0-22b4-4f7c-8b20-b30b115f32db service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:06.485362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 672/1349] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:11.465044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77985f22-084e-4ad0-89ca-03c9446fb125 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:11.465626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77985f22-084e-4ad0-89ca-03c9446fb125 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:11.465863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77985f22-084e-4ad0-89ca-03c9446fb125 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:11.469152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 678/1350] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:29:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:11.475090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d4383-f512-4258-a5a7-843c0f092b77 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:11.475597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d4383-f512-4258-a5a7-843c0f092b77 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:11.475597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d4383-f512-4258-a5a7-843c0f092b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:11.566712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-639d4383-f512-4258-a5a7-843c0f092b77 None None] MFA Rules not processed for user `4b181d38588d44d4bb72fa319d2ba2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:11.568246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-639d4383-f512-4258-a5a7-843c0f092b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:11.800684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 673/1351] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:29:11 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:15.828601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=59 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:16.338716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73495cb3-8392-4745-a6d1-a34dc6cdf3ce None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:16.339342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73495cb3-8392-4745-a6d1-a34dc6cdf3ce None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:16.339852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73495cb3-8392-4745-a6d1-a34dc6cdf3ce None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:16.341753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 679/1352] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:29:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:16.347870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5d86b-220e-4355-acbf-07bb9c67a860 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:16.348386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5d86b-220e-4355-acbf-07bb9c67a860 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:16.348470 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5d86b-220e-4355-acbf-07bb9c67a860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:16.412203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:16.441926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-00e5d86b-220e-4355-acbf-07bb9c67a860 None None] MFA Rules not processed for user `4b181d38588d44d4bb72fa319d2ba2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:16.443388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-00e5d86b-220e-4355-acbf-07bb9c67a860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:16.573911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 674/1353] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:29:16 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:17.973021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddab7eb-962a-4bd6-8253-200650ef6cc4 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:17.973713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddab7eb-962a-4bd6-8253-200650ef6cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:17.973824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddab7eb-962a-4bd6-8253-200650ef6cc4 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:17.975443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 680/1354] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:29:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:17.981026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452af984-a31a-4ae8-b819-4f12e248a42d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:17.981547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452af984-a31a-4ae8-b819-4f12e248a42d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:17.981672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452af984-a31a-4ae8-b819-4f12e248a42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:18.073372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-452af984-a31a-4ae8-b819-4f12e248a42d None None] MFA Rules not processed for user `4b181d38588d44d4bb72fa319d2ba2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:18.074658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-452af984-a31a-4ae8-b819-4f12e248a42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:18.122107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:18.196786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 675/1355] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:29:17 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:21.833110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b23b43-9220-4cb6-b523-2efb19efa8bd None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:21.833902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b23b43-9220-4cb6-b523-2efb19efa8bd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:21.833902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b23b43-9220-4cb6-b523-2efb19efa8bd None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:21.835093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 681/1356] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:21 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:29:21.841933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1badd-4a71-46e4-a9c3-610f1823ba79 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:21.842486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1badd-4a71-46e4-a9c3-610f1823ba79 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:21.842486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1badd-4a71-46e4-a9c3-610f1823ba79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:21.925711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c9f1badd-4a71-46e4-a9c3-610f1823ba79 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:21.927011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c9f1badd-4a71-46e4-a9c3-610f1823ba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:22.090632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 676/1357] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:22.523417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e63499-721e-4fc1-bad3-cf7ea5418b3f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:22.524432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e63499-721e-4fc1-bad3-cf7ea5418b3f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:22.524932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e63499-721e-4fc1-bad3-cf7ea5418b3f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:22.526821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 682/1358] 10.222.0.100 () {54 vars in 967 bytes} [Thu Sep 1 17:29:22 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:22.536970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7e809-ebef-493a-9e1d-61321b2f9b34 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:22.538202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7e809-ebef-493a-9e1d-61321b2f9b34 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:22.539706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7e809-ebef-493a-9e1d-61321b2f9b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:22.569426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:22.670706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2ff7e809-ebef-493a-9e1d-61321b2f9b34 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:22.672768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2ff7e809-ebef-493a-9e1d-61321b2f9b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:22.833331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 677/1359] 10.222.0.100 () {58 vars in 1064 bytes} [Thu Sep 1 17:29:22 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:22.841785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e63499-721e-4fc1-bad3-cf7ea5418b3f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:22.844933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91e63499-721e-4fc1-bad3-cf7ea5418b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:23.049300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e63499-721e-4fc1-bad3-cf7ea5418b3f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:23.143081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:23.198266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42661dd-f562-4094-a49b-13c0e8114af6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:23.198875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42661dd-f562-4094-a49b-13c0e8114af6 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:23.199156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42661dd-f562-4094-a49b-13c0e8114af6 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:23.199426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42661dd-f562-4094-a49b-13c0e8114af6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:23.201068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d42661dd-f562-4094-a49b-13c0e8114af6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:23.318537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42661dd-f562-4094-a49b-13c0e8114af6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:23.320100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42661dd-f562-4094-a49b-13c0e8114af6 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:23.344500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 683/1360] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:25.606473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d98abe-5f24-4423-9dca-ef4eaefc6e08 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:25.607018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d98abe-5f24-4423-9dca-ef4eaefc6e08 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:25.607018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d98abe-5f24-4423-9dca-ef4eaefc6e08 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:25.609049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 678/1361] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:29:25.615573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfba96e-6a2e-4e62-b4ff-cb428583f972 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:25.616097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfba96e-6a2e-4e62-b4ff-cb428583f972 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:25.616292 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfba96e-6a2e-4e62-b4ff-cb428583f972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:25.713975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2bfba96e-6a2e-4e62-b4ff-cb428583f972 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:25.715260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2bfba96e-6a2e-4e62-b4ff-cb428583f972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:25.763748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:25.848689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 684/1362] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:26.168036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d98abe-5f24-4423-9dca-ef4eaefc6e08 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:26.169464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-11d98abe-5f24-4423-9dca-ef4eaefc6e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:26.283463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:26.400859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d98abe-5f24-4423-9dca-ef4eaefc6e08 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:26.613536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d869d3-60f4-4cde-b81f-a8f53091cf21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:26.614586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d869d3-60f4-4cde-b81f-a8f53091cf21 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:26.615089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d869d3-60f4-4cde-b81f-a8f53091cf21 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:26.615565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d869d3-60f4-4cde-b81f-a8f53091cf21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:26.619013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-42d869d3-60f4-4cde-b81f-a8f53091cf21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:26.671032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:26.777845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d869d3-60f4-4cde-b81f-a8f53091cf21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:26.779720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d869d3-60f4-4cde-b81f-a8f53091cf21 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:26.805765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 679/1363] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:29.444869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3281c2-2519-4c54-b918-9a8edf1c7512 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:29.445442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3281c2-2519-4c54-b918-9a8edf1c7512 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:29.445442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3281c2-2519-4c54-b918-9a8edf1c7512 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:29.447044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 685/1364] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:29.453350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda2f84-370b-4be1-823f-41f00d06b392 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:29.454235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda2f84-370b-4be1-823f-41f00d06b392 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:29.454235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda2f84-370b-4be1-823f-41f00d06b392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:29.537361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bdda2f84-370b-4be1-823f-41f00d06b392 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:29.538541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bdda2f84-370b-4be1-823f-41f00d06b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:29.693639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:29.705868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 680/1365] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:30.057371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8433c2-1309-4688-8536-ef6d02d55831 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:30.058113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8433c2-1309-4688-8536-ef6d02d55831 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:30.058733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8433c2-1309-4688-8536-ef6d02d55831 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:30.062446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 686/1366] 10.222.0.100 () {54 vars in 967 bytes} [Thu Sep 1 17:29:30 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:30.068709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1099fe7-d1bd-4f79-a325-2caaf989bd9c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:30.069006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1099fe7-d1bd-4f79-a325-2caaf989bd9c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:30.069294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1099fe7-d1bd-4f79-a325-2caaf989bd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:30.183800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f1099fe7-d1bd-4f79-a325-2caaf989bd9c None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:30.185040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f1099fe7-d1bd-4f79-a325-2caaf989bd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:30.361938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 681/1367] 10.222.0.100 () {58 vars in 1064 bytes} [Thu Sep 1 17:29:30 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:30.371507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8433c2-1309-4688-8536-ef6d02d55831 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:30.373184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ef8433c2-1309-4688-8536-ef6d02d55831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:30.597012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8433c2-1309-4688-8536-ef6d02d55831 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:30.626074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:30.776904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad36dd0-287c-437c-8f68-2b97fcf3ac4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:30.777882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad36dd0-287c-437c-8f68-2b97fcf3ac4f service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:30.778377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad36dd0-287c-437c-8f68-2b97fcf3ac4f service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:30.778842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad36dd0-287c-437c-8f68-2b97fcf3ac4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:30.781529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1ad36dd0-287c-437c-8f68-2b97fcf3ac4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:30.910892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad36dd0-287c-437c-8f68-2b97fcf3ac4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:30.912755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad36dd0-287c-437c-8f68-2b97fcf3ac4f service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:30.950697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 687/1368] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:31.102557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6f1c-25b5-49c4-abe2-aea672c9601a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:31.103583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6f1c-25b5-49c4-abe2-aea672c9601a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:31.103583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6f1c-25b5-49c4-abe2-aea672c9601a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:31.105427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 682/1369] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:31.112911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310f26e-600e-4c24-a5f1-3f8cbfe2c2db None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:31.113364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310f26e-600e-4c24-a5f1-3f8cbfe2c2db None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:31.113780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310f26e-600e-4c24-a5f1-3f8cbfe2c2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:31.158942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:31.212236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1310f26e-600e-4c24-a5f1-3f8cbfe2c2db None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:31.214170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1310f26e-600e-4c24-a5f1-3f8cbfe2c2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:31.353623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 688/1370] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:31.956785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d412fb-9abf-47c7-aaf2-912311c1c0cb None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:31.957224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d412fb-9abf-47c7-aaf2-912311c1c0cb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:31.957743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d412fb-9abf-47c7-aaf2-912311c1c0cb None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:31.960386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 683/1371] 10.222.0.100 () {54 vars in 969 bytes} [Thu Sep 1 17:29:31 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:31.966121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbb4ab-a3ff-4770-82c1-aecbfd2bf916 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:31.967315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbb4ab-a3ff-4770-82c1-aecbfd2bf916 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:31.968315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbb4ab-a3ff-4770-82c1-aecbfd2bf916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:32.075727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4bdbb4ab-a3ff-4770-82c1-aecbfd2bf916 None None] MFA Rules not processed for user `1d47d8e089f446d3bb4b6609d7256ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:32.077653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4bdbb4ab-a3ff-4770-82c1-aecbfd2bf916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:32.175442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:32.230032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 689/1372] 10.222.0.100 () {58 vars in 1066 bytes} [Thu Sep 1 17:29:31 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:32.243574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d412fb-9abf-47c7-aaf2-912311c1c0cb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:32.246367 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-15d412fb-9abf-47c7-aaf2-912311c1c0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:32.327607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:32.422693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d412fb-9abf-47c7-aaf2-912311c1c0cb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:32.555363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46301a06-75e6-48e3-8230-50f6eae482d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:32.556154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46301a06-75e6-48e3-8230-50f6eae482d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:32.556950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46301a06-75e6-48e3-8230-50f6eae482d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:32.556950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46301a06-75e6-48e3-8230-50f6eae482d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:32.562338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-46301a06-75e6-48e3-8230-50f6eae482d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:32.682007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:32.696511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46301a06-75e6-48e3-8230-50f6eae482d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:32.698020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46301a06-75e6-48e3-8230-50f6eae482d5 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:32.716385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 684/1373] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:33.206303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a870-6342-44b1-ba3b-5ab791686568 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:33.206850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a870-6342-44b1-ba3b-5ab791686568 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:33.206850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a870-6342-44b1-ba3b-5ab791686568 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:33.208220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 690/1374] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:33.213894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460d139-c419-4e9b-b854-ed2506bc6dd0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:33.214263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460d139-c419-4e9b-b854-ed2506bc6dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:33.214667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460d139-c419-4e9b-b854-ed2506bc6dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:33.294405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9460d139-c419-4e9b-b854-ed2506bc6dd0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:33.296210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9460d139-c419-4e9b-b854-ed2506bc6dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:33.424841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 685/1375] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:33.746749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3a870-6342-44b1-ba3b-5ab791686568 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:33.748278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c4d3a870-6342-44b1-ba3b-5ab791686568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:33.930025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3a870-6342-44b1-ba3b-5ab791686568 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:34.010390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:34.078784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934db75-b2fe-4f6f-9ddc-665c27312221 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:34.079397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934db75-b2fe-4f6f-9ddc-665c27312221 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:34.079675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934db75-b2fe-4f6f-9ddc-665c27312221 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:34.079943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934db75-b2fe-4f6f-9ddc-665c27312221 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:34.081612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6934db75-b2fe-4f6f-9ddc-665c27312221 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:34.187574 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934db75-b2fe-4f6f-9ddc-665c27312221 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:34.189122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934db75-b2fe-4f6f-9ddc-665c27312221 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:34.214261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 691/1376] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:36.261921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fcf6e-b7c9-4d13-b400-0f1864e06e48 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:36.262471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fcf6e-b7c9-4d13-b400-0f1864e06e48 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:36.262471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fcf6e-b7c9-4d13-b400-0f1864e06e48 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:36.263847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 686/1377] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:36.271528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4303a-6d56-43b1-a6bb-c5de5b7c019e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:36.272311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4303a-6d56-43b1-a6bb-c5de5b7c019e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:36.272587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4303a-6d56-43b1-a6bb-c5de5b7c019e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:36.342522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d7b4303a-6d56-43b1-a6bb-c5de5b7c019e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:36.343751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d7b4303a-6d56-43b1-a6bb-c5de5b7c019e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:36.360037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:36.486147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 692/1378] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:36.798259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4fcf6e-b7c9-4d13-b400-0f1864e06e48 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:36.799836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9a4fcf6e-b7c9-4d13-b400-0f1864e06e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:36.865884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:36.965112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4fcf6e-b7c9-4d13-b400-0f1864e06e48 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:37.093454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e4616-6e98-41e1-a93e-f209e4d66ff7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:37.094082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e4616-6e98-41e1-a93e-f209e4d66ff7 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:37.094356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e4616-6e98-41e1-a93e-f209e4d66ff7 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:37.094632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e4616-6e98-41e1-a93e-f209e4d66ff7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:37.096201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-266e4616-6e98-41e1-a93e-f209e4d66ff7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:37.116344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:37.200159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e4616-6e98-41e1-a93e-f209e4d66ff7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:37.202799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e4616-6e98-41e1-a93e-f209e4d66ff7 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:37.224741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 687/1379] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:39.223012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac303db-e06d-4ff0-a44d-ed76c0f41e97 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:39.223577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac303db-e06d-4ff0-a44d-ed76c0f41e97 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:39.223577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac303db-e06d-4ff0-a44d-ed76c0f41e97 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:39.225186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 693/1380] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:39.230612 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc1d3f-a7f1-46fe-8e5a-b5d68d4fff73 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:39.231079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc1d3f-a7f1-46fe-8e5a-b5d68d4fff73 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:39.231183 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc1d3f-a7f1-46fe-8e5a-b5d68d4fff73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:39.313152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2bbc1d3f-a7f1-46fe-8e5a-b5d68d4fff73 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:39.314906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2bbc1d3f-a7f1-46fe-8e5a-b5d68d4fff73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:39.438429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:39.461599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 688/1381] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:39.756668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac303db-e06d-4ff0-a44d-ed76c0f41e97 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:39.758653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8ac303db-e06d-4ff0-a44d-ed76c0f41e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:39.907227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:39.919688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac303db-e06d-4ff0-a44d-ed76c0f41e97 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:40.059811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89f0c5-d439-4d97-9a9f-bea675b78f3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:40.060417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89f0c5-d439-4d97-9a9f-bea675b78f3a service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:40.060698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89f0c5-d439-4d97-9a9f-bea675b78f3a service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:40.061596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89f0c5-d439-4d97-9a9f-bea675b78f3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:40.063167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ac89f0c5-d439-4d97-9a9f-bea675b78f3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:40.173422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89f0c5-d439-4d97-9a9f-bea675b78f3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:40.174991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89f0c5-d439-4d97-9a9f-bea675b78f3a service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:40.186706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:40.196148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 694/1382] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:42.410142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d92ba-5643-49d7-892b-0278c6782f27 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:42.410757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d92ba-5643-49d7-892b-0278c6782f27 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:42.410757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d92ba-5643-49d7-892b-0278c6782f27 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:42.412102 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 689/1383] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:29:42.417875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72c00b-48c8-4639-8f2c-3bbf86c271bc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:42.418471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72c00b-48c8-4639-8f2c-3bbf86c271bc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:42.418471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72c00b-48c8-4639-8f2c-3bbf86c271bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:42.498288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0a72c00b-48c8-4639-8f2c-3bbf86c271bc None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:42.500328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0a72c00b-48c8-4639-8f2c-3bbf86c271bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:42.647175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 695/1384] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:29:43.077901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d92ba-5643-49d7-892b-0278c6782f27 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:43.079471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cd0d92ba-5643-49d7-892b-0278c6782f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:43.234104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d92ba-5643-49d7-892b-0278c6782f27 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:43.305075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:43.357686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f89a9d-3ef9-4bc7-ba07-2cc3c13a3c5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:43.358784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f89a9d-3ef9-4bc7-ba07-2cc3c13a3c5d service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:43.359056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f89a9d-3ef9-4bc7-ba07-2cc3c13a3c5d service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:43.359324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f89a9d-3ef9-4bc7-ba07-2cc3c13a3c5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:43.360935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e3f89a9d-3ef9-4bc7-ba07-2cc3c13a3c5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:43.457175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f89a9d-3ef9-4bc7-ba07-2cc3c13a3c5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:43.458893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f89a9d-3ef9-4bc7-ba07-2cc3c13a3c5d service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:43.478377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 690/1385] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:43 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:45.457265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb30f5-dadf-42f8-a239-08e6e0714aca None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:45.457866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb30f5-dadf-42f8-a239-08e6e0714aca None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:45.457866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb30f5-dadf-42f8-a239-08e6e0714aca None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:45.459229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 696/1386] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:45.465267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ff94f-8a45-4bee-8d7f-c6f4ad3b8458 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:45.465795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ff94f-8a45-4bee-8d7f-c6f4ad3b8458 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:45.466051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ff94f-8a45-4bee-8d7f-c6f4ad3b8458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:45.553633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-361ff94f-8a45-4bee-8d7f-c6f4ad3b8458 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:45.554890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-361ff94f-8a45-4bee-8d7f-c6f4ad3b8458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:45.600079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:45.689226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 691/1387] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:45.995840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb30f5-dadf-42f8-a239-08e6e0714aca None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:45.997410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4dbb30f5-dadf-42f8-a239-08e6e0714aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:46.096604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:46.214290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb30f5-dadf-42f8-a239-08e6e0714aca None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:46.398041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eec87a-7037-4cdb-85d6-dfc1cecc4dad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:46.398664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eec87a-7037-4cdb-85d6-dfc1cecc4dad service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:46.398941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eec87a-7037-4cdb-85d6-dfc1cecc4dad service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:46.399208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eec87a-7037-4cdb-85d6-dfc1cecc4dad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:46.400915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d1eec87a-7037-4cdb-85d6-dfc1cecc4dad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:46.449251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:46.526865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eec87a-7037-4cdb-85d6-dfc1cecc4dad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:46.528504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eec87a-7037-4cdb-85d6-dfc1cecc4dad service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:46.550998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 697/1388] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:48.171445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14dd13-35ed-4853-97dc-c1226f57573a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:48.172080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14dd13-35ed-4853-97dc-c1226f57573a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:48.172080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14dd13-35ed-4853-97dc-c1226f57573a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:48.173333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 692/1389] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:48.178821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ffe84-5499-4995-af37-c32ae968faf4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:48.179310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ffe84-5499-4995-af37-c32ae968faf4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:48.179420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ffe84-5499-4995-af37-c32ae968faf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:48.248913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3b8ffe84-5499-4995-af37-c32ae968faf4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:48.250255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3b8ffe84-5499-4995-af37-c32ae968faf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:48.367504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 698/1390] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:48.677321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d66f2f-26b7-4473-9495-9a1c9345b253 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:48.677640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d66f2f-26b7-4473-9495-9a1c9345b253 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:48.677896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d66f2f-26b7-4473-9495-9a1c9345b253 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:48.679233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 693/1391] 10.222.0.100 () {54 vars in 972 bytes} [Thu Sep 1 17:29:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:48.686081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c301d-61a8-4f4a-bb9b-b2623d78f668 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:48.686717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c301d-61a8-4f4a-bb9b-b2623d78f668 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:48.687065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c301d-61a8-4f4a-bb9b-b2623d78f668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:48.706653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:48.783266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a54c301d-61a8-4f4a-bb9b-b2623d78f668 None None] MFA Rules not processed for user `e0891d81e6e24ad99a9b532059c8c2af`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:48.784274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270ea9-6f90-413f-ba61-2896975e80df None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:48.784491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a54c301d-61a8-4f4a-bb9b-b2623d78f668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:48.784629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270ea9-6f90-413f-ba61-2896975e80df None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:48.784817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270ea9-6f90-413f-ba61-2896975e80df None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:48.786206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 694/1392] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:29:48.792269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585296f8-2fcb-4a9f-88a4-b6e37c404c70 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:48.792269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585296f8-2fcb-4a9f-88a4-b6e37c404c70 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:48.792577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585296f8-2fcb-4a9f-88a4-b6e37c404c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:48.868915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-585296f8-2fcb-4a9f-88a4-b6e37c404c70 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:48.869336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 699/1393] 10.222.0.100 () {58 vars in 1069 bytes} [Thu Sep 1 17:29:48 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:48.870247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-585296f8-2fcb-4a9f-88a4-b6e37c404c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:48.877018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c301d-61a8-4f4a-bb9b-b2623d78f668 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:48.885503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c301d-61a8-4f4a-bb9b-b2623d78f668 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:48.980923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52d95d-7b73-45f1-9aa5-cab9189a1f40 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0891d81e6e24ad99a9b532059c8c2af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:48.981450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52d95d-7b73-45f1-9aa5-cab9189a1f40 service glance] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:48.981761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52d95d-7b73-45f1-9aa5-cab9189a1f40 service glance] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:48.982037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52d95d-7b73-45f1-9aa5-cab9189a1f40 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:48.983801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ea52d95d-7b73-45f1-9aa5-cab9189a1f40 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:48.989724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 695/1394] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:29:49.015927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:49.099912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52d95d-7b73-45f1-9aa5-cab9189a1f40 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:49.101340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52d95d-7b73-45f1-9aa5-cab9189a1f40 service glance] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:49.119645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 700/1395] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:29:49.287468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585296f8-2fcb-4a9f-88a4-b6e37c404c70 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:49.288917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-585296f8-2fcb-4a9f-88a4-b6e37c404c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:49.303559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:49.413639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585296f8-2fcb-4a9f-88a4-b6e37c404c70 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:49.537493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46070e6-fabb-4740-b79f-41f54d45b9c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:49.538037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46070e6-fabb-4740-b79f-41f54d45b9c1 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:49.538317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46070e6-fabb-4740-b79f-41f54d45b9c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:49.538589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46070e6-fabb-4740-b79f-41f54d45b9c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:49.540208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c46070e6-fabb-4740-b79f-41f54d45b9c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:49.552212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:49.643354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46070e6-fabb-4740-b79f-41f54d45b9c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:49.644992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46070e6-fabb-4740-b79f-41f54d45b9c1 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:49.665130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 696/1396] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:49.887517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306ed44-537b-4cbf-afb2-7fbf1968b6ec None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:49.888018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306ed44-537b-4cbf-afb2-7fbf1968b6ec None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:49.888121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306ed44-537b-4cbf-afb2-7fbf1968b6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:49.959329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1306ed44-537b-4cbf-afb2-7fbf1968b6ec None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:49.960551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1306ed44-537b-4cbf-afb2-7fbf1968b6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:50.043240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:50.050847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 701/1397] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:29:49 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:50.094341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7066f0-358f-455a-972f-13dd96bb3d87 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:50.094633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7066f0-358f-455a-972f-13dd96bb3d87 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:50.094908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7066f0-358f-455a-972f-13dd96bb3d87 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:50.096265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 697/1398] 10.222.0.100 () {54 vars in 948 bytes} [Thu Sep 1 17:29:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:50.102764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200666be-86ee-4939-b293-d7ae651790de None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:50.103120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200666be-86ee-4939-b293-d7ae651790de None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:50.103461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200666be-86ee-4939-b293-d7ae651790de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:50.185299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-200666be-86ee-4939-b293-d7ae651790de None None] MFA Rules not processed for user `f3fa2499f5594b7798f6fe4443a14f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:50.186736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-200666be-86ee-4939-b293-d7ae651790de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:50.296854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 702/1399] 10.222.0.100 () {58 vars in 1045 bytes} [Thu Sep 1 17:29:50 2022] POST /identity/v3/auth/tokens => generated 3314 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:50.306288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7066f0-358f-455a-972f-13dd96bb3d87 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:50.307850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5b7066f0-358f-455a-972f-13dd96bb3d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:50.444194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:50.446149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7066f0-358f-455a-972f-13dd96bb3d87 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:50.553472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c32f1-d200-4bf3-8141-5b370a07e41a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:50.554114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c32f1-d200-4bf3-8141-5b370a07e41a service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:50.554387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c32f1-d200-4bf3-8141-5b370a07e41a service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:50.554653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c32f1-d200-4bf3-8141-5b370a07e41a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:50.556304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a03c32f1-d200-4bf3-8141-5b370a07e41a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:50.641979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c32f1-d200-4bf3-8141-5b370a07e41a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:50.643609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c32f1-d200-4bf3-8141-5b370a07e41a service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:50.651988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 698/1400] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:29:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:29:51.662232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662ef56-c91d-4e55-b004-fc0f20f700b1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:51.662728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662ef56-c91d-4e55-b004-fc0f20f700b1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:51.662845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662ef56-c91d-4e55-b004-fc0f20f700b1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:51.664219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 703/1401] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:51 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:51.670447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9bb1-a41d-4cc9-8167-403cb65f0740 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:51.670957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9bb1-a41d-4cc9-8167-403cb65f0740 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:51.671069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9bb1-a41d-4cc9-8167-403cb65f0740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:51.713895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:51.744242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fd8a9bb1-a41d-4cc9-8167-403cb65f0740 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:51.745458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fd8a9bb1-a41d-4cc9-8167-403cb65f0740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:51.854296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 699/1402] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:52.149058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662ef56-c91d-4e55-b004-fc0f20f700b1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:52.150649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e662ef56-c91d-4e55-b004-fc0f20f700b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:52.229734 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:52.330447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662ef56-c91d-4e55-b004-fc0f20f700b1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:52.476294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f7938-0eec-47c0-9e45-9911b07f9af6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:52.476893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f7938-0eec-47c0-9e45-9911b07f9af6 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:52.477185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f7938-0eec-47c0-9e45-9911b07f9af6 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:52.477461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f7938-0eec-47c0-9e45-9911b07f9af6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:52.479394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e69f7938-0eec-47c0-9e45-9911b07f9af6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:52.498501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:52.593662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f7938-0eec-47c0-9e45-9911b07f9af6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:52.595272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f7938-0eec-47c0-9e45-9911b07f9af6 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:52.629641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 704/1403] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:54.678948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa473d-8487-42d9-9644-d2c8660b3fc6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:54.679926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa473d-8487-42d9-9644-d2c8660b3fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:54.680468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa473d-8487-42d9-9644-d2c8660b3fc6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:54.685359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 700/1404] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:54 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:54.687238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba81cd9-b775-4030-8c8d-254200a5c73e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:54.687700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba81cd9-b775-4030-8c8d-254200a5c73e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:54.687807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba81cd9-b775-4030-8c8d-254200a5c73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:54.769846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6ba81cd9-b775-4030-8c8d-254200a5c73e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:54.771084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6ba81cd9-b775-4030-8c8d-254200a5c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:54.878217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:54.889947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 705/1405] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:55.180310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa473d-8487-42d9-9644-d2c8660b3fc6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:55.182345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4eaa473d-8487-42d9-9644-d2c8660b3fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:55.311100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:55.359661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa473d-8487-42d9-9644-d2c8660b3fc6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:55.488970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386ccfc-838f-49eb-b841-7e49fc6d0b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:55.489598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386ccfc-838f-49eb-b841-7e49fc6d0b64 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:55.489956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386ccfc-838f-49eb-b841-7e49fc6d0b64 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:55.490231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386ccfc-838f-49eb-b841-7e49fc6d0b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:55.491898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a386ccfc-838f-49eb-b841-7e49fc6d0b64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:55.577212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:55.596827 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386ccfc-838f-49eb-b841-7e49fc6d0b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:55.598483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386ccfc-838f-49eb-b841-7e49fc6d0b64 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:55.616621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 701/1406] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:29:57.621431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54c2c-3c49-42b6-ad3f-80f870e69c0d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:57.622039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54c2c-3c49-42b6-ad3f-80f870e69c0d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:57.622039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54c2c-3c49-42b6-ad3f-80f870e69c0d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:57.623983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 706/1407] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:29:57 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:29:57.629435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92d6ee-3eec-4533-b0fc-caa5dc1dce72 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:57.629909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92d6ee-3eec-4533-b0fc-caa5dc1dce72 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:57.630084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92d6ee-3eec-4533-b0fc-caa5dc1dce72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:57.711564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2a92d6ee-3eec-4533-b0fc-caa5dc1dce72 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:29:57.712910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2a92d6ee-3eec-4533-b0fc-caa5dc1dce72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:57.933050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 702/1408] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:29:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:29:58.229139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b54c2c-3c49-42b6-ad3f-80f870e69c0d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:29:58.230778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-67b54c2c-3c49-42b6-ad3f-80f870e69c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:58.403546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b54c2c-3c49-42b6-ad3f-80f870e69c0d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:29:58.481460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:29:58.536414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e534a-5f9f-4a8a-955d-02e58574ed1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:29:58.537126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e534a-5f9f-4a8a-955d-02e58574ed1e service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:29:58.537467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e534a-5f9f-4a8a-955d-02e58574ed1e service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:29:58.537827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e534a-5f9f-4a8a-955d-02e58574ed1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:29:58.539430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-837e534a-5f9f-4a8a-955d-02e58574ed1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:29:58.642354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e534a-5f9f-4a8a-955d-02e58574ed1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:29:58.643949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e534a-5f9f-4a8a-955d-02e58574ed1e service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:29:58.663906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 707/1409] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:29:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:07.931514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=78 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:08.661400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=65 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:13.373545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915956a0-b940-4a62-9f9a-2e69c6071297 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:13.374108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915956a0-b940-4a62-9f9a-2e69c6071297 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:13.374108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915956a0-b940-4a62-9f9a-2e69c6071297 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:13.375452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 703/1410] 10.222.0.100 () {54 vars in 923 bytes} [Thu Sep 1 17:30:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:13.380767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc60367-f34c-46e5-bfa4-041d68ef996a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:13.381296 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc60367-f34c-46e5-bfa4-041d68ef996a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:13.381403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc60367-f34c-46e5-bfa4-041d68ef996a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:13.460009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6dc60367-f34c-46e5-bfa4-041d68ef996a None None] MFA Rules not processed for user `4b181d38588d44d4bb72fa319d2ba2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:13.461226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6dc60367-f34c-46e5-bfa4-041d68ef996a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:13.573961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 708/1411] 10.222.0.100 () {58 vars in 1020 bytes} [Thu Sep 1 17:30:13 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:13.589157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f99707-b758-4c12-8de9-a1383dd047ca None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:13.589444 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f99707-b758-4c12-8de9-a1383dd047ca None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:13.589753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f99707-b758-4c12-8de9-a1383dd047ca None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:13.591114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 704/1412] 10.222.0.100 () {54 vars in 923 bytes} [Thu Sep 1 17:30:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:13.596051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6f07-2832-4cc0-af74-ee9a0313ac66 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:13.596341 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6f07-2832-4cc0-af74-ee9a0313ac66 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:13.596616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6f07-2832-4cc0-af74-ee9a0313ac66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:13.677692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:13.680540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2a9d6f07-2832-4cc0-af74-ee9a0313ac66 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:13.682249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2a9d6f07-2832-4cc0-af74-ee9a0313ac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:13.800343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 709/1413] 10.222.0.100 () {58 vars in 1020 bytes} [Thu Sep 1 17:30:13 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:16.209904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3785a-62da-4822-9456-fd7539aec73f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:16.210459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3785a-62da-4822-9456-fd7539aec73f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:16.210459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3785a-62da-4822-9456-fd7539aec73f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:16.211813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:16.217202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c462e11-0b3d-4944-b704-321a9ab2d9a3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:16.217684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c462e11-0b3d-4944-b704-321a9ab2d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:16.217790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c462e11-0b3d-4944-b704-321a9ab2d9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:16.285390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2c462e11-0b3d-4944-b704-321a9ab2d9a3 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:16.286649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2c462e11-0b3d-4944-b704-321a9ab2d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:16.343705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:16.392236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:16.684750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3785a-62da-4822-9456-fd7539aec73f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:16.687012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c6e3785a-62da-4822-9456-fd7539aec73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:16.835058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3785a-62da-4822-9456-fd7539aec73f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:16.908550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:16.948668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57b396-ab89-46d9-8344-09814cb1c6eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:16.949276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57b396-ab89-46d9-8344-09814cb1c6eb service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:16.949555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57b396-ab89-46d9-8344-09814cb1c6eb service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:16.949857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57b396-ab89-46d9-8344-09814cb1c6eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:16.951898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dd57b396-ab89-46d9-8344-09814cb1c6eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:17.041852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57b396-ab89-46d9-8344-09814cb1c6eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:17.043438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57b396-ab89-46d9-8344-09814cb1c6eb service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:17.061979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 706/1416] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:17.984625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e405f7e-c6c5-4df1-a2ec-e633799f1135 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:17.985197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e405f7e-c6c5-4df1-a2ec-e633799f1135 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:17.985197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e405f7e-c6c5-4df1-a2ec-e633799f1135 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:17.986467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:30:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:17.991653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bfadd-190d-4fce-8687-319e320953dc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:17.992139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bfadd-190d-4fce-8687-319e320953dc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:17.992244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bfadd-190d-4fce-8687-319e320953dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:18.074196 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8e7bfadd-190d-4fce-8687-319e320953dc None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:18.075935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8e7bfadd-190d-4fce-8687-319e320953dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:18.104326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:18.178667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:30:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:18.334380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e405f7e-c6c5-4df1-a2ec-e633799f1135 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:18.336311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1e405f7e-c6c5-4df1-a2ec-e633799f1135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:18.486413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e405f7e-c6c5-4df1-a2ec-e633799f1135 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:18.530828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:18.612731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0ec4f-3e35-4f48-b1a1-417cec343d03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:18.613269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0ec4f-3e35-4f48-b1a1-417cec343d03 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:18.613547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0ec4f-3e35-4f48-b1a1-417cec343d03 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:18.613861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0ec4f-3e35-4f48-b1a1-417cec343d03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:18.615491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-85b0ec4f-3e35-4f48-b1a1-417cec343d03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:18.709650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0ec4f-3e35-4f48-b1a1-417cec343d03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:18.711235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0ec4f-3e35-4f48-b1a1-417cec343d03 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:18.729540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 712/1419] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:19.647085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934698d-1a8c-4eab-928b-374f11d17f8c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:19.647655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934698d-1a8c-4eab-928b-374f11d17f8c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:19.647655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934698d-1a8c-4eab-928b-374f11d17f8c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:19.648991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:30:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:19.655570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e85074-21c4-4b24-b553-ab18cb8735c6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:19.656126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e85074-21c4-4b24-b553-ab18cb8735c6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:19.656126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e85074-21c4-4b24-b553-ab18cb8735c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:19.720778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:19.746337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-17e85074-21c4-4b24-b553-ab18cb8735c6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:19.748200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-17e85074-21c4-4b24-b553-ab18cb8735c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:19.853425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:30:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:20.016863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934698d-1a8c-4eab-928b-374f11d17f8c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:20.018501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5934698d-1a8c-4eab-928b-374f11d17f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:20.176484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934698d-1a8c-4eab-928b-374f11d17f8c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:20.189676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:20.292213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b18c1-854f-49b9-83f5-463d28ca7e20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:20.292727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b18c1-854f-49b9-83f5-463d28ca7e20 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:20.293002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b18c1-854f-49b9-83f5-463d28ca7e20 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:20.293267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b18c1-854f-49b9-83f5-463d28ca7e20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:20.295841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ff3b18c1-854f-49b9-83f5-463d28ca7e20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:20.391132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b18c1-854f-49b9-83f5-463d28ca7e20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:20.392736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b18c1-854f-49b9-83f5-463d28ca7e20 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:20.413219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 709/1422] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:21.340470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf1996-ee77-4a0c-8132-7e901411c13e None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:21.341505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf1996-ee77-4a0c-8132-7e901411c13e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:21.341864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf1996-ee77-4a0c-8132-7e901411c13e None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:21.343443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:30:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:21.348887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e1125-9ec6-47ee-b1e6-33568419dd45 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:21.349447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e1125-9ec6-47ee-b1e6-33568419dd45 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:21.349447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e1125-9ec6-47ee-b1e6-33568419dd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:21.371417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:21.439002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a34e1125-9ec6-47ee-b1e6-33568419dd45 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:21.440246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a34e1125-9ec6-47ee-b1e6-33568419dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:21.561013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:30:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:21.716759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf1996-ee77-4a0c-8132-7e901411c13e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:21.718352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-85cf1996-ee77-4a0c-8132-7e901411c13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:21.830760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:21.868259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf1996-ee77-4a0c-8132-7e901411c13e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:21.984746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a3f47-cbc8-4c45-9356-ff095b7085d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:21.985277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a3f47-cbc8-4c45-9356-ff095b7085d6 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:21.985559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a3f47-cbc8-4c45-9356-ff095b7085d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:21.985867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a3f47-cbc8-4c45-9356-ff095b7085d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:21.987482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-735a3f47-cbc8-4c45-9356-ff095b7085d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:22.064427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:22.078965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a3f47-cbc8-4c45-9356-ff095b7085d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:22.080585 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a3f47-cbc8-4c45-9356-ff095b7085d6 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:22.099702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 715/1425] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:22.986665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4f9eb-7cb7-4d26-87ce-a67d8ed41a3d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:22.987192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4f9eb-7cb7-4d26-87ce-a67d8ed41a3d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:22.987192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4f9eb-7cb7-4d26-87ce-a67d8ed41a3d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:22.988492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:30:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:22.993599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dba63-3689-4cc6-9790-2c134d5a4afe None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:22.994342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dba63-3689-4cc6-9790-2c134d5a4afe None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:22.994612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dba63-3689-4cc6-9790-2c134d5a4afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:23.086619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5e1dba63-3689-4cc6-9790-2c134d5a4afe None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:23.087839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5e1dba63-3689-4cc6-9790-2c134d5a4afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:23.195775 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:30:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:23.359051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4f9eb-7cb7-4d26-87ce-a67d8ed41a3d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:23.360579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eaa4f9eb-7cb7-4d26-87ce-a67d8ed41a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:23.457670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:23.531828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4f9eb-7cb7-4d26-87ce-a67d8ed41a3d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:23.667157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd22021-a678-43b0-bed0-ee298b32d1f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:23.667772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd22021-a678-43b0-bed0-ee298b32d1f0 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:23.668056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd22021-a678-43b0-bed0-ee298b32d1f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:23.668325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd22021-a678-43b0-bed0-ee298b32d1f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:23.670196 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5fd22021-a678-43b0-bed0-ee298b32d1f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:23.732721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:23.778857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd22021-a678-43b0-bed0-ee298b32d1f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:23.780530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd22021-a678-43b0-bed0-ee298b32d1f0 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:23.801192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 712/1428] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:24.656794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04aac6-59e1-4a28-9bb8-75836ac664e0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:24.657312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04aac6-59e1-4a28-9bb8-75836ac664e0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:24.657424 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04aac6-59e1-4a28-9bb8-75836ac664e0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:24.659042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:30:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:24.664851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c504-72d4-48a7-917b-3cf5f3f13a29 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:24.665385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c504-72d4-48a7-917b-3cf5f3f13a29 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:24.665385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c504-72d4-48a7-917b-3cf5f3f13a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:24.767074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2013c504-72d4-48a7-917b-3cf5f3f13a29 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:24.768412 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2013c504-72d4-48a7-917b-3cf5f3f13a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:24.884172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:30:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:25.042597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04aac6-59e1-4a28-9bb8-75836ac664e0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:25.044834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4b04aac6-59e1-4a28-9bb8-75836ac664e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:25.101953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:25.215468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04aac6-59e1-4a28-9bb8-75836ac664e0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:25.353146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092adc5-6198-4706-8484-a5dcef3abed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:25.353706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092adc5-6198-4706-8484-a5dcef3abed7 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:25.353991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092adc5-6198-4706-8484-a5dcef3abed7 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:25.354269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092adc5-6198-4706-8484-a5dcef3abed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:25.355960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a092adc5-6198-4706-8484-a5dcef3abed7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:25.364719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:25.472307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092adc5-6198-4706-8484-a5dcef3abed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:25.473976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092adc5-6198-4706-8484-a5dcef3abed7 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:25.493267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 718/1431] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:26.387201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0df35-445b-4df8-910c-bab21986877c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:26.387803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0df35-445b-4df8-910c-bab21986877c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:26.387917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0df35-445b-4df8-910c-bab21986877c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:26.389526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:30:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:26.403090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd44116-99a1-494b-85e4-22297a2c0b1b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:26.403649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd44116-99a1-494b-85e4-22297a2c0b1b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:26.403649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd44116-99a1-494b-85e4-22297a2c0b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:26.489598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-efd44116-99a1-494b-85e4-22297a2c0b1b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:26.490820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-efd44116-99a1-494b-85e4-22297a2c0b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:26.566258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:26.609525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:30:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:26.765328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0df35-445b-4df8-910c-bab21986877c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:26.766826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2ad0df35-445b-4df8-910c-bab21986877c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:26.788010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:26.926017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0df35-445b-4df8-910c-bab21986877c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:27.028290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:27.056259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c9802-66ea-45cb-a4f0-1ce4f7c96f80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:27.056861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c9802-66ea-45cb-a4f0-1ce4f7c96f80 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:27.057139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c9802-66ea-45cb-a4f0-1ce4f7c96f80 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:27.057409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c9802-66ea-45cb-a4f0-1ce4f7c96f80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:27.059060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e21c9802-66ea-45cb-a4f0-1ce4f7c96f80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:27.156713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c9802-66ea-45cb-a4f0-1ce4f7c96f80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:27.158654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c9802-66ea-45cb-a4f0-1ce4f7c96f80 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:27.186245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 715/1434] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:28.051358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0077c-e288-4ea6-adcd-83a9f8ef6c82 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:28.051925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0077c-e288-4ea6-adcd-83a9f8ef6c82 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:28.051925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0077c-e288-4ea6-adcd-83a9f8ef6c82 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:28.053175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:30:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:28.058630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7352561-34ba-474b-b1d0-3e9820c8b03f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:28.059154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7352561-34ba-474b-b1d0-3e9820c8b03f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:28.059154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7352561-34ba-474b-b1d0-3e9820c8b03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:28.151249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d7352561-34ba-474b-b1d0-3e9820c8b03f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:28.152918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d7352561-34ba-474b-b1d0-3e9820c8b03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:28.198631 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:28.259293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:30:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:28.417198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0077c-e288-4ea6-adcd-83a9f8ef6c82 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:28.419332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-76f0077c-e288-4ea6-adcd-83a9f8ef6c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:28.607260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0077c-e288-4ea6-adcd-83a9f8ef6c82 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:28.678860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:28.756548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534efc33-d9ca-438d-8f5d-5bc16e331930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:28.757067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534efc33-d9ca-438d-8f5d-5bc16e331930 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:28.757349 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534efc33-d9ca-438d-8f5d-5bc16e331930 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:28.757645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534efc33-d9ca-438d-8f5d-5bc16e331930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:28.759293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-534efc33-d9ca-438d-8f5d-5bc16e331930 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:28.898012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534efc33-d9ca-438d-8f5d-5bc16e331930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:28.901237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534efc33-d9ca-438d-8f5d-5bc16e331930 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:28.930560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 721/1437] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:29.836077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064669e-4825-4c2b-95e7-715889c66509 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:29.836722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064669e-4825-4c2b-95e7-715889c66509 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:29.836722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064669e-4825-4c2b-95e7-715889c66509 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:29.838034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:30:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:30:29.844446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01495ba-c6c8-4f75-a727-283ff0e49542 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:29.845289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01495ba-c6c8-4f75-a727-283ff0e49542 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:29.845455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01495ba-c6c8-4f75-a727-283ff0e49542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:29.952329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d01495ba-c6c8-4f75-a727-283ff0e49542 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:29.953534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d01495ba-c6c8-4f75-a727-283ff0e49542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:29.956038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:30.081618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:30:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:30.236888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064669e-4825-4c2b-95e7-715889c66509 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:30.238461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b064669e-4825-4c2b-95e7-715889c66509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:30.424669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064669e-4825-4c2b-95e7-715889c66509 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:30.461614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:30.590867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f6566-30cf-4e77-8ed7-42c20d7b97d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:30.591358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6566-30cf-4e77-8ed7-42c20d7b97d7 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:30.591643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6566-30cf-4e77-8ed7-42c20d7b97d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:30.591909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6566-30cf-4e77-8ed7-42c20d7b97d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:30.593567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d57f6566-30cf-4e77-8ed7-42c20d7b97d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:30.710704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f6566-30cf-4e77-8ed7-42c20d7b97d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:30.712512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f6566-30cf-4e77-8ed7-42c20d7b97d7 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:30.733497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 718/1440] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:32.681753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c6bd1-fa36-4041-b9b3-f5f852bc636c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:32.682303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c6bd1-fa36-4041-b9b3-f5f852bc636c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:32.682303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c6bd1-fa36-4041-b9b3-f5f852bc636c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:32.683646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:32.690131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97507b-b305-49ee-a497-b835c2ae49bb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:32.690706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97507b-b305-49ee-a497-b835c2ae49bb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:32.690706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97507b-b305-49ee-a497-b835c2ae49bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:32.736264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:32.767249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bf97507b-b305-49ee-a497-b835c2ae49bb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:32.768474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bf97507b-b305-49ee-a497-b835c2ae49bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:32.881302 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:33.178622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c6bd1-fa36-4041-b9b3-f5f852bc636c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:33.180163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-603c6bd1-fa36-4041-b9b3-f5f852bc636c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:33.308130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c6bd1-fa36-4041-b9b3-f5f852bc636c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:33.342577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:33.391741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e5203-6236-4a1b-b641-c5c4d82feeea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:33.392358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e5203-6236-4a1b-b641-c5c4d82feeea service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:33.392767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e5203-6236-4a1b-b641-c5c4d82feeea service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:33.393241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e5203-6236-4a1b-b641-c5c4d82feeea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:33.396416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-940e5203-6236-4a1b-b641-c5c4d82feeea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:33.530126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e5203-6236-4a1b-b641-c5c4d82feeea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:33.531501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e5203-6236-4a1b-b641-c5c4d82feeea service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:33.554858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 724/1443] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:30:35.715272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e37e4-d202-4b7d-9235-8d779b924659 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:35.715892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e37e4-d202-4b7d-9235-8d779b924659 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:35.715892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e37e4-d202-4b7d-9235-8d779b924659 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:35.717358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:35.723736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5743811-a88d-4016-aabf-957de76ef0ec None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:35.724771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5743811-a88d-4016-aabf-957de76ef0ec None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:35.724771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5743811-a88d-4016-aabf-957de76ef0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:35.801034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b5743811-a88d-4016-aabf-957de76ef0ec None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:35.802283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b5743811-a88d-4016-aabf-957de76ef0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:35.818912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:35.933786 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:36.219747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e37e4-d202-4b7d-9235-8d779b924659 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:36.221837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-da6e37e4-d202-4b7d-9235-8d779b924659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:36.344103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e37e4-d202-4b7d-9235-8d779b924659 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:36.358165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:36.434931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77048b7f-e0cd-437b-816d-c2640ed645ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:36.435543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77048b7f-e0cd-437b-816d-c2640ed645ad service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:36.435824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77048b7f-e0cd-437b-816d-c2640ed645ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:36.436149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77048b7f-e0cd-437b-816d-c2640ed645ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:36.437784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-77048b7f-e0cd-437b-816d-c2640ed645ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:36.559824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77048b7f-e0cd-437b-816d-c2640ed645ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:36.561408 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77048b7f-e0cd-437b-816d-c2640ed645ad service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:36.581255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 721/1446] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:38.978319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a42bd-d663-4614-a9a1-c8d88f6bf3b7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:38.978864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a42bd-d663-4614-a9a1-c8d88f6bf3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:38.978864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a42bd-d663-4614-a9a1-c8d88f6bf3b7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:38.980521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:30:38.987292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852446f5-693f-49e5-844d-b105a2e9faa4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:38.987807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852446f5-693f-49e5-844d-b105a2e9faa4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:38.987916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852446f5-693f-49e5-844d-b105a2e9faa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:39.039041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:39.059077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-852446f5-693f-49e5-844d-b105a2e9faa4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:39.061185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-852446f5-693f-49e5-844d-b105a2e9faa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:39.181478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:39.469605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a42bd-d663-4614-a9a1-c8d88f6bf3b7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:39.471228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ff0a42bd-d663-4614-a9a1-c8d88f6bf3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:39.581977 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a42bd-d663-4614-a9a1-c8d88f6bf3b7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:39.630901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:39.667483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a700a3-05de-4952-8cdb-4bcbdd9752d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:39.668166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a700a3-05de-4952-8cdb-4bcbdd9752d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:39.668364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a700a3-05de-4952-8cdb-4bcbdd9752d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:39.668809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a700a3-05de-4952-8cdb-4bcbdd9752d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:39.670286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-15a700a3-05de-4952-8cdb-4bcbdd9752d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:39.790421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a700a3-05de-4952-8cdb-4bcbdd9752d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:39.791763 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a700a3-05de-4952-8cdb-4bcbdd9752d3 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:39.810830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 727/1449] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:41.912500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4abb48-96c5-447b-b85d-e920f49e622b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:41.913066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4abb48-96c5-447b-b85d-e920f49e622b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:41.913066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4abb48-96c5-447b-b85d-e920f49e622b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:41.914477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:41.920040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8d409-c1fb-4ac5-9da5-2e41f384f857 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:41.920483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8d409-c1fb-4ac5-9da5-2e41f384f857 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:41.920629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8d409-c1fb-4ac5-9da5-2e41f384f857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:41.992289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-27d8d409-c1fb-4ac5-9da5-2e41f384f857 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:41.993521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-27d8d409-c1fb-4ac5-9da5-2e41f384f857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:42.016165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:42.093312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:42.381723 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4abb48-96c5-447b-b85d-e920f49e622b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:42.383498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0b4abb48-96c5-447b-b85d-e920f49e622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:42.508791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4abb48-96c5-447b-b85d-e920f49e622b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:42.530752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:42.595461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baacc9-6963-4e33-aa83-96c7ff45ef6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:42.596078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baacc9-6963-4e33-aa83-96c7ff45ef6b service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:42.596355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baacc9-6963-4e33-aa83-96c7ff45ef6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:42.596627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baacc9-6963-4e33-aa83-96c7ff45ef6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:42.598336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b0baacc9-6963-4e33-aa83-96c7ff45ef6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:42.724721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0baacc9-6963-4e33-aa83-96c7ff45ef6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:42.726205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0baacc9-6963-4e33-aa83-96c7ff45ef6b service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:42.748243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 724/1452] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:44.847444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f78c2-957e-4793-b0b9-083d7770696f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:44.848027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f78c2-957e-4793-b0b9-083d7770696f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:44.848027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f78c2-957e-4793-b0b9-083d7770696f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:44.849420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:44.855267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1302-6579-46bd-befb-123c854a15b9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:44.856588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1302-6579-46bd-befb-123c854a15b9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:44.856588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1302-6579-46bd-befb-123c854a15b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:44.928469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:44.931478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4a3d1302-6579-46bd-befb-123c854a15b9 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:44.933533 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4a3d1302-6579-46bd-befb-123c854a15b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:45.054401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:45.341440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f78c2-957e-4793-b0b9-083d7770696f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:45.343025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cf2f78c2-957e-4793-b0b9-083d7770696f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:45.461209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f78c2-957e-4793-b0b9-083d7770696f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:45.518336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:45.546117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b42a4d-cd05-418b-ad99-fa66afe1c975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:45.546731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b42a4d-cd05-418b-ad99-fa66afe1c975 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:45.547004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b42a4d-cd05-418b-ad99-fa66afe1c975 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:45.547272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b42a4d-cd05-418b-ad99-fa66afe1c975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:45.548921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-42b42a4d-cd05-418b-ad99-fa66afe1c975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:45.676447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b42a4d-cd05-418b-ad99-fa66afe1c975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:45.677959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b42a4d-cd05-418b-ad99-fa66afe1c975 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:45.697930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 730/1455] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:48.021975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cbb59-7f5d-4a54-ba46-7b527a647805 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:48.022560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cbb59-7f5d-4a54-ba46-7b527a647805 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:48.022560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cbb59-7f5d-4a54-ba46-7b527a647805 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:48.023930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:48.030508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23461aa-75fe-4321-a29f-10a6a508f344 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:48.031003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23461aa-75fe-4321-a29f-10a6a508f344 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:48.031174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23461aa-75fe-4321-a29f-10a6a508f344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:48.109692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d23461aa-75fe-4321-a29f-10a6a508f344 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:48.110924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d23461aa-75fe-4321-a29f-10a6a508f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:48.143043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:48.217965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:30:48.525044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cbb59-7f5d-4a54-ba46-7b527a647805 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:48.528598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-063cbb59-7f5d-4a54-ba46-7b527a647805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:48.652113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cbb59-7f5d-4a54-ba46-7b527a647805 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:48.685167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:48.743069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234f12-92c2-4c68-aec8-4bed137c7455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:48.743682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234f12-92c2-4c68-aec8-4bed137c7455 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:48.743957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234f12-92c2-4c68-aec8-4bed137c7455 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:48.744225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234f12-92c2-4c68-aec8-4bed137c7455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:48.746049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-86234f12-92c2-4c68-aec8-4bed137c7455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:48.873716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86234f12-92c2-4c68-aec8-4bed137c7455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:48.875216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86234f12-92c2-4c68-aec8-4bed137c7455 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:48.896347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 727/1458] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:50.963894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a580c-dd58-4d75-be09-f5d769e8eb71 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:50.964475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a580c-dd58-4d75-be09-f5d769e8eb71 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:50.964475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a580c-dd58-4d75-be09-f5d769e8eb71 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:50.965880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:50.971154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4047c-a4b4-4383-8ad6-87af25430827 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:50.971560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4047c-a4b4-4383-8ad6-87af25430827 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:50.971707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4047c-a4b4-4383-8ad6-87af25430827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:51.039099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e7f4047c-a4b4-4383-8ad6-87af25430827 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:51.040320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e7f4047c-a4b4-4383-8ad6-87af25430827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:51.049835 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:51.144726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:51.457893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a580c-dd58-4d75-be09-f5d769e8eb71 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:51.459460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-eb6a580c-dd58-4d75-be09-f5d769e8eb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:51.569803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a580c-dd58-4d75-be09-f5d769e8eb71 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:51.627287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:51.650762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7526f8-09f8-40f1-bfee-d0ff4ff8a2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:51.651715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7526f8-09f8-40f1-bfee-d0ff4ff8a2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:51.652165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7526f8-09f8-40f1-bfee-d0ff4ff8a2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:51.653292 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7526f8-09f8-40f1-bfee-d0ff4ff8a2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:51.655743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6a7526f8-09f8-40f1-bfee-d0ff4ff8a2b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:51.771015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7526f8-09f8-40f1-bfee-d0ff4ff8a2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:51.772353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7526f8-09f8-40f1-bfee-d0ff4ff8a2b2 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:51.792177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 733/1461] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:53.861879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977438-9bb1-4bc3-bf12-ea7679006d00 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:53.862430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977438-9bb1-4bc3-bf12-ea7679006d00 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:53.862430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977438-9bb1-4bc3-bf12-ea7679006d00 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:53.863702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:53.869157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97493043-2371-406e-b8c6-20243d522c6c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:53.869669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97493043-2371-406e-b8c6-20243d522c6c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:53.869779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97493043-2371-406e-b8c6-20243d522c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:53.941133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-97493043-2371-406e-b8c6-20243d522c6c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:53.942375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-97493043-2371-406e-b8c6-20243d522c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:53.983336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:54.044978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:54.333702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63977438-9bb1-4bc3-bf12-ea7679006d00 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:54.335233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-63977438-9bb1-4bc3-bf12-ea7679006d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:54.441043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63977438-9bb1-4bc3-bf12-ea7679006d00 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:54.479482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:54.516197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5af0cd-89be-4e50-8457-6fb835dd2b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:54.516787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5af0cd-89be-4e50-8457-6fb835dd2b55 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:54.517065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5af0cd-89be-4e50-8457-6fb835dd2b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:54.517334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5af0cd-89be-4e50-8457-6fb835dd2b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:54.518980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ef5af0cd-89be-4e50-8457-6fb835dd2b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:54.628854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5af0cd-89be-4e50-8457-6fb835dd2b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:54.630238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5af0cd-89be-4e50-8457-6fb835dd2b55 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:54.650523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 730/1464] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:56.679223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad53aa3-e3ba-4927-8f7b-3c134abe424b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:56.679917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad53aa3-e3ba-4927-8f7b-3c134abe424b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:56.679917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad53aa3-e3ba-4927-8f7b-3c134abe424b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:56.681042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:56.687355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17fb50-4ae2-4354-9770-cc3f6883f53f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:56.687855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17fb50-4ae2-4354-9770-cc3f6883f53f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:56.687962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17fb50-4ae2-4354-9770-cc3f6883f53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:56.768780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ba17fb50-4ae2-4354-9770-cc3f6883f53f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:56.770064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ba17fb50-4ae2-4354-9770-cc3f6883f53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:56.795108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:56.886224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:30:57.178316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad53aa3-e3ba-4927-8f7b-3c134abe424b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:30:57.179880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1ad53aa3-e3ba-4927-8f7b-3c134abe424b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:57.290981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad53aa3-e3ba-4927-8f7b-3c134abe424b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:30:57.368216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:57.376061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e221221-0962-49de-b8c8-2d1758e8780a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:30:57.376668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e221221-0962-49de-b8c8-2d1758e8780a service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:57.377057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e221221-0962-49de-b8c8-2d1758e8780a service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:57.377326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e221221-0962-49de-b8c8-2d1758e8780a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:57.378898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3e221221-0962-49de-b8c8-2d1758e8780a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:57.494439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e221221-0962-49de-b8c8-2d1758e8780a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:30:57.495788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e221221-0962-49de-b8c8-2d1758e8780a service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:30:57.517171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 736/1467] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:30:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:30:59.544796 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ceac3-6441-4ed8-b81a-b004e42bc93a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:59.545394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ceac3-6441-4ed8-b81a-b004e42bc93a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:59.545394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ceac3-6441-4ed8-b81a-b004e42bc93a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:59.546734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:30:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:30:59.552550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200f02-f29a-4b79-8487-251e6f084347 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:30:59.553120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200f02-f29a-4b79-8487-251e6f084347 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:30:59.553120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200f02-f29a-4b79-8487-251e6f084347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:30:59.628385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3c200f02-f29a-4b79-8487-251e6f084347 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:30:59.629647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3c200f02-f29a-4b79-8487-251e6f084347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:30:59.685356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:30:59.740091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:30:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:00.035361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ceac3-6441-4ed8-b81a-b004e42bc93a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:00.037072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f49ceac3-6441-4ed8-b81a-b004e42bc93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:00.150533 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ceac3-6441-4ed8-b81a-b004e42bc93a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:00.202496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:00.231349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb2777-554b-46d8-8316-4a6b298b19fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:00.232258 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2777-554b-46d8-8316-4a6b298b19fb service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:00.232258 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2777-554b-46d8-8316-4a6b298b19fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:00.232503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2777-554b-46d8-8316-4a6b298b19fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:00.234120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0afb2777-554b-46d8-8316-4a6b298b19fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:00.358990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb2777-554b-46d8-8316-4a6b298b19fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:00.360473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb2777-554b-46d8-8316-4a6b298b19fb service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:00.381154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 733/1470] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:09.737886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=22 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:10.378565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=66 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:20.608428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424671d-31bd-478d-a33f-1a4c15df72f8 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:20.609067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424671d-31bd-478d-a33f-1a4c15df72f8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:20.609067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424671d-31bd-478d-a33f-1a4c15df72f8 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:20.610466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 738/1471] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:31:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:20.615966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8cfaf-9a97-44b2-a89d-49bff714df14 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:20.616468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8cfaf-9a97-44b2-a89d-49bff714df14 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:20.616575 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8cfaf-9a97-44b2-a89d-49bff714df14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:20.687921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f6b8cfaf-9a97-44b2-a89d-49bff714df14 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:20.689073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f6b8cfaf-9a97-44b2-a89d-49bff714df14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:20.789564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 734/1472] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:31:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:21.086762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f1415-bbff-44ed-aed4-23f8a6dd207d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:21.087045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f1415-bbff-44ed-aed4-23f8a6dd207d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:21.087315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f1415-bbff-44ed-aed4-23f8a6dd207d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:21.088687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 739/1473] 10.222.0.100 () {54 vars in 972 bytes} [Thu Sep 1 17:31:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:21.094195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db458-80d0-4bc5-870a-24be5009bde1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:21.094484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db458-80d0-4bc5-870a-24be5009bde1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:21.094755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db458-80d0-4bc5-870a-24be5009bde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:21.289874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-243db458-80d0-4bc5-870a-24be5009bde1 None None] MFA Rules not processed for user `e0891d81e6e24ad99a9b532059c8c2af`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:21.290998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-243db458-80d0-4bc5-870a-24be5009bde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:21.311595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:21.366040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 735/1474] 10.222.0.100 () {58 vars in 1069 bytes} [Thu Sep 1 17:31:21 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:21.374213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f1415-bbff-44ed-aed4-23f8a6dd207d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:21.375889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-303f1415-bbff-44ed-aed4-23f8a6dd207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:21.483429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f1415-bbff-44ed-aed4-23f8a6dd207d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:21.558553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cd258-e784-45b3-9f6c-bf1a358411c4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0891d81e6e24ad99a9b532059c8c2af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:21.559080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cd258-e784-45b3-9f6c-bf1a358411c4 service glance] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:21.559361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cd258-e784-45b3-9f6c-bf1a358411c4 service glance] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:21.559639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cd258-e784-45b3-9f6c-bf1a358411c4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:21.561250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8c1cd258-e784-45b3-9f6c-bf1a358411c4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:21.615853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:21.671987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cd258-e784-45b3-9f6c-bf1a358411c4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:21.673431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cd258-e784-45b3-9f6c-bf1a358411c4 service glance] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:21.692734 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 740/1475] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:23.708277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5723d5b-1dab-4bcb-b91e-ce01192099f4 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:23.708839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5723d5b-1dab-4bcb-b91e-ce01192099f4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:23.708839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5723d5b-1dab-4bcb-b91e-ce01192099f4 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:23.710153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 736/1476] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:31:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:23.715502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da617de7-5c9c-46ac-9a3b-14e167c5e3c8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:23.715940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da617de7-5c9c-46ac-9a3b-14e167c5e3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:23.716063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da617de7-5c9c-46ac-9a3b-14e167c5e3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:23.784909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-da617de7-5c9c-46ac-9a3b-14e167c5e3c8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:23.786047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-da617de7-5c9c-46ac-9a3b-14e167c5e3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:23.883512 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 741/1477] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:31:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:24.159286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5723d5b-1dab-4bcb-b91e-ce01192099f4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:24.160822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d5723d5b-1dab-4bcb-b91e-ce01192099f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:24.299452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5723d5b-1dab-4bcb-b91e-ce01192099f4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:24.347908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:24.392219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806c126-fa84-4c61-91aa-810bbf50f473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:24.392814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806c126-fa84-4c61-91aa-810bbf50f473 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:24.393094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806c126-fa84-4c61-91aa-810bbf50f473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:24.393359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806c126-fa84-4c61-91aa-810bbf50f473 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:24.394022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806c126-fa84-4c61-91aa-810bbf50f473 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:24.395741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-7806c126-fa84-4c61-91aa-810bbf50f473 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:31:24.395741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Sep 01 17:31:24.395741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: Sep 01 17:31:24.396870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806c126-fa84-4c61-91aa-810bbf50f473 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:24.415919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 737/1478] 10.222.0.100 () {56 vars in 1113 bytes} [Thu Sep 1 17:31:24 2022] GET /identity/v3/projects => generated 2562 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:31:26.398268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182a215-575b-4b5d-9c83-a85e8ad03d1b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.398880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182a215-575b-4b5d-9c83-a85e8ad03d1b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.398880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182a215-575b-4b5d-9c83-a85e8ad03d1b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.400200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 742/1479] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:31:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:26.405804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70806e4-d545-49db-8a0f-07538deb28ec None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:26.406256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70806e4-d545-49db-8a0f-07538deb28ec None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:26.406370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70806e4-d545-49db-8a0f-07538deb28ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:26.479596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f70806e4-d545-49db-8a0f-07538deb28ec None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:26.481344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f70806e4-d545-49db-8a0f-07538deb28ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:26.583308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 738/1480] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:31:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:26.882944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182a215-575b-4b5d-9c83-a85e8ad03d1b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:26.884660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b182a215-575b-4b5d-9c83-a85e8ad03d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:26.895680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:27.038952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182a215-575b-4b5d-9c83-a85e8ad03d1b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:27.237593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:27.272057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04929a5-571f-4f64-abd1-34797b01a371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:27.272663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04929a5-571f-4f64-abd1-34797b01a371 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:27.273545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04929a5-571f-4f64-abd1-34797b01a371 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:27.273853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04929a5-571f-4f64-abd1-34797b01a371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:27.275481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b04929a5-571f-4f64-abd1-34797b01a371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:27.365611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04929a5-571f-4f64-abd1-34797b01a371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:27.367190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04929a5-571f-4f64-abd1-34797b01a371 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:27.385367 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 743/1481] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:29.536990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ae425-2351-4a38-9227-9884b4bd6ae8 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:29.537608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ae425-2351-4a38-9227-9884b4bd6ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:29.537608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ae425-2351-4a38-9227-9884b4bd6ae8 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:29.539062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 739/1482] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:31:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:29.544654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8aafc-680e-40c7-a95a-b4cff687a550 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:29.545128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8aafc-680e-40c7-a95a-b4cff687a550 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:29.545244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8aafc-680e-40c7-a95a-b4cff687a550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:29.610685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8db8aafc-680e-40c7-a95a-b4cff687a550 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:29.611875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8db8aafc-680e-40c7-a95a-b4cff687a550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:29.660490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:29.713408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 744/1483] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:31:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:30.007855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ae425-2351-4a38-9227-9884b4bd6ae8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:30.009423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ef9ae425-2351-4a38-9227-9884b4bd6ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:30.035072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:30.118058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ae425-2351-4a38-9227-9884b4bd6ae8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:30.196403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0571cb9-473e-4d30-8ab1-18d0fd279262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:30.196999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0571cb9-473e-4d30-8ab1-18d0fd279262 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:30.197276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0571cb9-473e-4d30-8ab1-18d0fd279262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:30.197545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0571cb9-473e-4d30-8ab1-18d0fd279262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:30.200313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f0571cb9-473e-4d30-8ab1-18d0fd279262 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:30.274754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:30.311287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0571cb9-473e-4d30-8ab1-18d0fd279262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:30.312714 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0571cb9-473e-4d30-8ab1-18d0fd279262 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:30.331622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 740/1484] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:32.325843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb65be6-3fc8-4cae-88c0-d7c1e5d6c341 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:32.326340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb65be6-3fc8-4cae-88c0-d7c1e5d6c341 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:32.326490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb65be6-3fc8-4cae-88c0-d7c1e5d6c341 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:32.327749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 745/1485] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:31:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:32.333105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee059a-73e6-4504-bd21-9a769c6547d6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:32.333596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee059a-73e6-4504-bd21-9a769c6547d6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:32.333708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee059a-73e6-4504-bd21-9a769c6547d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:32.406546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7fee059a-73e6-4504-bd21-9a769c6547d6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:32.407754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7fee059a-73e6-4504-bd21-9a769c6547d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:32.518705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 741/1486] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:31:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:32.819020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb65be6-3fc8-4cae-88c0-d7c1e5d6c341 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:32.820617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-edb65be6-3fc8-4cae-88c0-d7c1e5d6c341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:32.924330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb65be6-3fc8-4cae-88c0-d7c1e5d6c341 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:32.997962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:32.999910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534fecd-3cc8-4317-b3b8-3e5b88d41ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:33.000572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534fecd-3cc8-4317-b3b8-3e5b88d41ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:33.001068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534fecd-3cc8-4317-b3b8-3e5b88d41ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:33.001463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534fecd-3cc8-4317-b3b8-3e5b88d41ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:33.003535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6534fecd-3cc8-4317-b3b8-3e5b88d41ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:33.111801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534fecd-3cc8-4317-b3b8-3e5b88d41ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:33.113226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534fecd-3cc8-4317-b3b8-3e5b88d41ee0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:33.132215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 746/1487] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:35.234477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2f1f0-07f0-43e8-abc5-4178aa98f726 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.235057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2f1f0-07f0-43e8-abc5-4178aa98f726 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.235057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2f1f0-07f0-43e8-abc5-4178aa98f726 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.236365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 742/1488] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:31:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:35.242070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923c915-1255-460a-ad36-ad436fc5b5d7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.242529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923c915-1255-460a-ad36-ad436fc5b5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.242657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923c915-1255-460a-ad36-ad436fc5b5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.308860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d923c915-1255-460a-ad36-ad436fc5b5d7 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:35.310095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d923c915-1255-460a-ad36-ad436fc5b5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.362552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:35.407353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 747/1489] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:31:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:35.706329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2f1f0-07f0-43e8-abc5-4178aa98f726 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:35.708445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-42d2f1f0-07f0-43e8-abc5-4178aa98f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.746395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:35.816296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2f1f0-07f0-43e8-abc5-4178aa98f726 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:35.893125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8a91d-27d1-4423-932b-d4d137bdd270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:35.893760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8a91d-27d1-4423-932b-d4d137bdd270 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:35.894047 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8a91d-27d1-4423-932b-d4d137bdd270 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:35.894317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8a91d-27d1-4423-932b-d4d137bdd270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:35.895938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4bc8a91d-27d1-4423-932b-d4d137bdd270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:35.982926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:36.015831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8a91d-27d1-4423-932b-d4d137bdd270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:36.017278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8a91d-27d1-4423-932b-d4d137bdd270 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:36.036140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 743/1490] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:38.262772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb542fd4-dd73-4952-9871-cd8a060c106c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.263700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb542fd4-dd73-4952-9871-cd8a060c106c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.263700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb542fd4-dd73-4952-9871-cd8a060c106c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.265236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 748/1491] 10.222.0.100 () {54 vars in 932 bytes} [Thu Sep 1 17:31:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:31:38.270944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2aa5a5-3568-439a-9153-062344859b99 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.271422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2aa5a5-3568-439a-9153-062344859b99 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.271528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2aa5a5-3568-439a-9153-062344859b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.347014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bf2aa5a5-3568-439a-9153-062344859b99 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:31:38.348386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bf2aa5a5-3568-439a-9153-062344859b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.458989 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 744/1492] 10.222.0.100 () {58 vars in 1029 bytes} [Thu Sep 1 17:31:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:31:38.774130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb542fd4-dd73-4952-9871-cd8a060c106c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:31:38.775669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bb542fd4-dd73-4952-9871-cd8a060c106c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.880210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb542fd4-dd73-4952-9871-cd8a060c106c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:31:38.957612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccad3e3-156e-4e65-89b0-f89f887671d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:31:38.958134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad3e3-156e-4e65-89b0-f89f887671d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:31:38.958408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad3e3-156e-4e65-89b0-f89f887671d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:31:38.958671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad3e3-156e-4e65-89b0-f89f887671d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:31:38.960251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6ccad3e3-156e-4e65-89b0-f89f887671d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:31:38.969753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:39.076564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccad3e3-156e-4e65-89b0-f89f887671d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:31:39.078024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccad3e3-156e-4e65-89b0-f89f887671d8 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:31:39.097810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 749/1493] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:31:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:31:48.456890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=82 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:31:49.095558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=50 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:57.629083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8c2bf-5f54-459c-a4b4-6e81039184b2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:57.629922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8c2bf-5f54-459c-a4b4-6e81039184b2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:57.629922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8c2bf-5f54-459c-a4b4-6e81039184b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:57.707556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1ec8c2bf-5f54-459c-a4b4-6e81039184b2 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:57.708762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1ec8c2bf-5f54-459c-a4b4-6e81039184b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:57.807876 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 745/1494] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:32:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:57.817320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb97d4-d40b-4bfe-aff4-8acc06f1ada2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:57.817949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb97d4-d40b-4bfe-aff4-8acc06f1ada2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:57.817949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb97d4-d40b-4bfe-aff4-8acc06f1ada2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:57.896156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ecfb97d4-d40b-4bfe-aff4-8acc06f1ada2 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:57.897379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ecfb97d4-d40b-4bfe-aff4-8acc06f1ada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:58.003692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 750/1495] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:32:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:58.009073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8c2bf-5f54-459c-a4b4-6e81039184b2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:58.010419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1ec8c2bf-5f54-459c-a4b4-6e81039184b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:58.021678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8c2bf-5f54-459c-a4b4-6e81039184b2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:58.083571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:58.119344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:58.119942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:58.120220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:58.120488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:58.121123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:32:58.121418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:58.122446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:32:58.122446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Sep 01 17:32:58.122446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: Sep 01 17:32:58.123316 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:58.134854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 746/1496] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:32:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:32:58.143657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb97d4-d40b-4bfe-aff4-8acc06f1ada2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:58.153735 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb97d4-d40b-4bfe-aff4-8acc06f1ada2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:58.216325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:58.250112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:58.250719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:58.250992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:58.251254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:58.251891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:58.253673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:32:58.253673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Sep 01 17:32:58.253673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: Sep 01 17:32:58.254860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:58.313484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 751/1497] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:32:58 2022] POST /identity/v3/projects => generated 378 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:58.317782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:58.319413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:58.354323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d303d49-d0a6-4ec0-95e8-444e206524a7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:58.450324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:58.450853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:58.451131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:58.451405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:58.452081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:58.453099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:32:58.453099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Sep 01 17:32:58.453099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: Sep 01 17:32:58.454354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:58.498245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 747/1498] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:32:58 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:32:58.502392 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:58.503913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:58.569335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc262fef-754d-44ec-bb56-3ca36e3660c1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:58.652518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:58.667484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:58.668081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:58.668351 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:58.668611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:58.669259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:58.670271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:32:58.670271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Sep 01 17:32:58.670271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: Sep 01 17:32:58.671136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:58.683843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 752/1499] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:58 2022] GET /identity/v3/roles => generated 1427 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:58.689567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:58.691735 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:58.704315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3faa8c-9fe9-4ef1-8263-40d0e803ec1f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:58.731464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:58.813028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849314d6-3599-4307-b10c-c252573f02eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:58.813675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849314d6-3599-4307-b10c-c252573f02eb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:58.813962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849314d6-3599-4307-b10c-c252573f02eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:58.814231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849314d6-3599-4307-b10c-c252573f02eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:58.814872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849314d6-3599-4307-b10c-c252573f02eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:58.815891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-849314d6-3599-4307-b10c-c252573f02eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:32:58.815891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Sep 01 17:32:58.815891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: Sep 01 17:32:58.816782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849314d6-3599-4307-b10c-c252573f02eb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:58.830121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 748/1500] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:32:58 2022] GET /identity/v3/roles => generated 1427 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:32:58.834599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:58.836221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:58.871607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f7bd3-c25d-4070-9f8b-f3a7baa290da admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:58.974926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edb364-68a1-4c38-9711-10e338db23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:58.975959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edb364-68a1-4c38-9711-10e338db23c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:58.976463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edb364-68a1-4c38-9711-10e338db23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:58.977479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edb364-68a1-4c38-9711-10e338db23c1 admin admin] PATH_INFO: `/v3/projects/7bc99f55987d4ce48b603c7cb9237a97/users/58e2d850b5ee4a05a6342df174d17e22/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:59.005652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edb364-68a1-4c38-9711-10e338db23c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc99f55987d4ce48b603c7cb9237a97, user_id=58e2d850b5ee4a05a6342df174d17e22, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:59.006686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-00edb364-68a1-4c38-9711-10e338db23c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:32:59.006686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Sep 01 17:32:59.006686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: Sep 01 17:32:59.008611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edb364-68a1-4c38-9711-10e338db23c1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:59.032814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 753/1501] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:32:58 2022] PUT /identity/v3/projects/7bc99f55987d4ce48b603c7cb9237a97/users/58e2d850b5ee4a05a6342df174d17e22/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:32:59.045825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957692a-2afe-402c-9545-6e26323914dc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:59.046106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957692a-2afe-402c-9545-6e26323914dc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:59.046376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957692a-2afe-402c-9545-6e26323914dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:59.103012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f957692a-2afe-402c-9545-6e26323914dc None None] MFA Rules not processed for user `58e2d850b5ee4a05a6342df174d17e22`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:59.104221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f957692a-2afe-402c-9545-6e26323914dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:59.182337 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 749/1502] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:32:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:59.195169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa168b-ae62-43dc-8944-0816d71bdf77 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:59.195462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa168b-ae62-43dc-8944-0816d71bdf77 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:59.195785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa168b-ae62-43dc-8944-0816d71bdf77 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:59.197117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 754/1503] 10.222.0.100 () {54 vars in 969 bytes} [Thu Sep 1 17:32:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:32:59.202034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8398889-d8fa-4653-a9c3-261ab0707dfe None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:59.202319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8398889-d8fa-4653-a9c3-261ab0707dfe None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:59.202591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8398889-d8fa-4653-a9c3-261ab0707dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:59.234070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:59.279432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a8398889-d8fa-4653-a9c3-261ab0707dfe None None] MFA Rules not processed for user `1d47d8e089f446d3bb4b6609d7256ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:32:59.280668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a8398889-d8fa-4653-a9c3-261ab0707dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:59.355615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 750/1504] 10.222.0.100 () {58 vars in 1066 bytes} [Thu Sep 1 17:32:59 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:32:59.363320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa168b-ae62-43dc-8944-0816d71bdf77 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:59.364822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-62aa168b-ae62-43dc-8944-0816d71bdf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:59.414591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:59.470805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa168b-ae62-43dc-8944-0816d71bdf77 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:59.546456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:59.546956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:59.547236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:59.547509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:59.549107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:59.648307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:59.650413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:59.654022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:32:59.669627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 755/1505] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:32:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:32:59.716971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c764c-fa23-48e8-9fc1-3d7c991ac440 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:59.717627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c764c-fa23-48e8-9fc1-3d7c991ac440 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:59.717627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c764c-fa23-48e8-9fc1-3d7c991ac440 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:59.718815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 751/1506] 10.222.0.100 () {52 vars in 835 bytes} [Thu Sep 1 17:32:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:32:59.722674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:32:59.723905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:32:59.828211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e641fe5-c032-4dea-a4d8-3603141efc33 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:32:59.946904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:32:59.947818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:32:59.948230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:32:59.948634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] PATH_INFO: `/v3/users/58e2d850b5ee4a05a6342df174d17e22` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:32:59.964011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58e2d850b5ee4a05a6342df174d17e22)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:32:59.965093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:32:59.965093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Sep 01 17:32:59.965093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: Sep 01 17:32:59.966358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:32:59.978129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:00.048417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58e2d850b5ee4a05a6342df174d17e22'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:00.059678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58e2d850b5ee4a05a6342df174d17e22'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:00.060306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58e2d850b5ee4a05a6342df174d17e22'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:00.085195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58e2d850b5ee4a05a6342df174d17e22'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:00.099870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58e2d850b5ee4a05a6342df174d17e22'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:00.105649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 756/1507] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:32:59 2022] DELETE /identity/v3/users/58e2d850b5ee4a05a6342df174d17e22 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:33:00.110202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2ee68-e9a9-4890-b426-81cde3829611 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:00.110481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2ee68-e9a9-4890-b426-81cde3829611 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:00.110777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2ee68-e9a9-4890-b426-81cde3829611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:00.173867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1ae2ee68-e9a9-4890-b426-81cde3829611 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:33:00.175094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1ae2ee68-e9a9-4890-b426-81cde3829611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:00.255977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:00.271057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 752/1508] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:33:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:33:00.281017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:00.282254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:00.384712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa86f0-fc8f-47a9-85c6-b258b9d06c76 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:00.441835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:00.462842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927be7c-04e9-42e7-a0f2-60f017f3084c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:00.463458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927be7c-04e9-42e7-a0f2-60f017f3084c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:00.463739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927be7c-04e9-42e7-a0f2-60f017f3084c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:00.464154 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927be7c-04e9-42e7-a0f2-60f017f3084c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:00.465654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d927be7c-04e9-42e7-a0f2-60f017f3084c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:00.576867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d927be7c-04e9-42e7-a0f2-60f017f3084c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:00.578341 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d927be7c-04e9-42e7-a0f2-60f017f3084c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:00.596118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 757/1509] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:33:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:33:00.958035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2ee68-e9a9-4890-b426-81cde3829611 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:33:00.959271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1ae2ee68-e9a9-4890-b426-81cde3829611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:33:00.972116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2ee68-e9a9-4890-b426-81cde3829611 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:33:01.080335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:33:01.080969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:33:01.081242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:33:01.081506 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] PATH_INFO: `/v3/projects/7bc99f55987d4ce48b603c7cb9237a97` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:33:01.097596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bc99f55987d4ce48b603c7cb9237a97)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:33:01.098622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:33:01.098622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Sep 01 17:33:01.098622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: Sep 01 17:33:01.099814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:33:01.172840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bc99f55987d4ce48b603c7cb9237a97', 'request_id': 'req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:01.184406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bc99f55987d4ce48b603c7cb9237a97', 'request_id': 'req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:01.190760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bc99f55987d4ce48b603c7cb9237a97', 'request_id': 'req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:01.222746 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] The token cache is being invalidate because project 7bc99f55987d4ce48b603c7cb9237a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:33:01.222943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:01.223500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9c349a1b-017d-4643-8429-ed3c7a6cc8d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:33:01.224833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 753/1510] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:33:00 2022] DELETE /identity/v3/projects/7bc99f55987d4ce48b603c7cb9237a97 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:33:10.593738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=62 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:33:11.220458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:30.406101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470e546-2f81-4e08-b12a-9d399e9f50b2 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:30.406710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470e546-2f81-4e08-b12a-9d399e9f50b2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:30.406710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470e546-2f81-4e08-b12a-9d399e9f50b2 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:30.408036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:34:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:34:30.413568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f89c0-3f50-4261-b260-b6978eb7cbf8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:30.414054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f89c0-3f50-4261-b260-b6978eb7cbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:30.414160 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f89c0-3f50-4261-b260-b6978eb7cbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:30.508566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2d7f89c0-3f50-4261-b260-b6978eb7cbf8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:30.510217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d7f89c0-3f50-4261-b260-b6978eb7cbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:30.633170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:34:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:30.792650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470e546-2f81-4e08-b12a-9d399e9f50b2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:30.794814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2470e546-2f81-4e08-b12a-9d399e9f50b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:30.913490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470e546-2f81-4e08-b12a-9d399e9f50b2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:30.919425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddf9e5-a156-445d-86bb-0d575465e2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:30.920036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddf9e5-a156-445d-86bb-0d575465e2bc service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:30.920320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddf9e5-a156-445d-86bb-0d575465e2bc service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:30.920591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddf9e5-a156-445d-86bb-0d575465e2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:30.929922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddf9e5-a156-445d-86bb-0d575465e2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:30.931641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddf9e5-a156-445d-86bb-0d575465e2bc service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:30.942353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 759/1513] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:31.772241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2271d-fc9e-4cc6-8e0e-883d5c46bdd1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:31.772804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2271d-fc9e-4cc6-8e0e-883d5c46bdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:31.772804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2271d-fc9e-4cc6-8e0e-883d5c46bdd1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:31.774197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:34:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:34:31.779095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f47dba-8599-4ac3-98c2-f4fb82755f4c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:31.779514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f47dba-8599-4ac3-98c2-f4fb82755f4c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:31.779631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f47dba-8599-4ac3-98c2-f4fb82755f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:31.799977 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d8f47dba-8599-4ac3-98c2-f4fb82755f4c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:31.801161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d8f47dba-8599-4ac3-98c2-f4fb82755f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:31.809985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:34:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:31.963268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2271d-fc9e-4cc6-8e0e-883d5c46bdd1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:31.977175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2271d-fc9e-4cc6-8e0e-883d5c46bdd1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:31.983467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a80e7-e56c-4a07-ba88-c1c4a49ed200 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:31.984056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a80e7-e56c-4a07-ba88-c1c4a49ed200 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:31.984331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a80e7-e56c-4a07-ba88-c1c4a49ed200 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:31.984673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a80e7-e56c-4a07-ba88-c1c4a49ed200 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:31.993100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a80e7-e56c-4a07-ba88-c1c4a49ed200 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:31.995557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a80e7-e56c-4a07-ba88-c1c4a49ed200 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:32.005284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 756/1516] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:32.607311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b220ce1-11c2-4813-905b-21f988d36dd2 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:32.607592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b220ce1-11c2-4813-905b-21f988d36dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:32.607867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b220ce1-11c2-4813-905b-21f988d36dd2 None None] PATH_INFO: `/v3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:32.609151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 761/1517] 10.222.0.17 () {54 vars in 929 bytes} [Thu Sep 1 17:34:32 2022] GET /identity/v3 => generated 256 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:34:32.635716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd4257-953a-436a-967f-fbbc806d7132 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:32.635984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd4257-953a-436a-967f-fbbc806d7132 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:32.636260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd4257-953a-436a-967f-fbbc806d7132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:32.699747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-35fd4257-953a-436a-967f-fbbc806d7132 None None] MFA Rules not processed for user `4b181d38588d44d4bb72fa319d2ba2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:32.700952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-35fd4257-953a-436a-967f-fbbc806d7132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:32.719425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:32.776700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 757/1518] 10.222.0.17 () {58 vars in 1019 bytes} [Thu Sep 1 17:34:32 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:36.686622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9e119-1a72-401f-9625-f100bcab9f06 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:36.687178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9e119-1a72-401f-9625-f100bcab9f06 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:36.687178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9e119-1a72-401f-9625-f100bcab9f06 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:36.688439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 762/1519] 10.222.0.100 () {54 vars in 966 bytes} [Thu Sep 1 17:34:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:34:36.693616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90c38f-ab9d-4f14-baaf-5f0da3994455 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:36.694124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90c38f-ab9d-4f14-baaf-5f0da3994455 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:36.694236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90c38f-ab9d-4f14-baaf-5f0da3994455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:36.715033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fb90c38f-ab9d-4f14-baaf-5f0da3994455 None None] MFA Rules not processed for user `4b181d38588d44d4bb72fa319d2ba2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:36.716254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fb90c38f-ab9d-4f14-baaf-5f0da3994455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:36.724947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 758/1520] 10.222.0.100 () {58 vars in 1063 bytes} [Thu Sep 1 17:34:36 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:36.733549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9e119-1a72-401f-9625-f100bcab9f06 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:36.747837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9e119-1a72-401f-9625-f100bcab9f06 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:36.752806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aabef8-f036-488d-88cc-e847378aafbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:36.753514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aabef8-f036-488d-88cc-e847378aafbe service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:36.753725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aabef8-f036-488d-88cc-e847378aafbe service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:36.754006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aabef8-f036-488d-88cc-e847378aafbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:36.763922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aabef8-f036-488d-88cc-e847378aafbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:36.765509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aabef8-f036-488d-88cc-e847378aafbe service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:36.775301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 763/1521] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:36 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:43.252673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064992-1db1-4035-aa74-180ed67d7663 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:43.253243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064992-1db1-4035-aa74-180ed67d7663 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:43.253243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064992-1db1-4035-aa74-180ed67d7663 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:43.254586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:34:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:34:43.259445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d06640-5cac-4943-8964-73e9b465d38e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:43.259881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d06640-5cac-4943-8964-73e9b465d38e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:43.260208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d06640-5cac-4943-8964-73e9b465d38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:43.279647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d9d06640-5cac-4943-8964-73e9b465d38e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:43.280842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d9d06640-5cac-4943-8964-73e9b465d38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:43.288965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:34:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:34:43.442223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8064992-1db1-4035-aa74-180ed67d7663 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:43.456788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8064992-1db1-4035-aa74-180ed67d7663 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:43.462304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091650cb-67e6-4784-b8f1-8d9b9a257c85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:43.462903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091650cb-67e6-4784-b8f1-8d9b9a257c85 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:43.463186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091650cb-67e6-4784-b8f1-8d9b9a257c85 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:43.464052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091650cb-67e6-4784-b8f1-8d9b9a257c85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:43.472347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091650cb-67e6-4784-b8f1-8d9b9a257c85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:43.473965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091650cb-67e6-4784-b8f1-8d9b9a257c85 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:43.483854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 760/1524] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:43 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:44.305670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c72f7f-2103-414f-8ce5-70a08a5d3690 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:44.306253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c72f7f-2103-414f-8ce5-70a08a5d3690 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:44.306253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c72f7f-2103-414f-8ce5-70a08a5d3690 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:44.308315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:34:44 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:34:44.313232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34176b74-d2de-4d53-827d-8b6589857660 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:44.313699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34176b74-d2de-4d53-827d-8b6589857660 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:44.313809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34176b74-d2de-4d53-827d-8b6589857660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:44.333639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-34176b74-d2de-4d53-827d-8b6589857660 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:44.334865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34176b74-d2de-4d53-827d-8b6589857660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:44.342698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:34:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:44.495351 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c72f7f-2103-414f-8ce5-70a08a5d3690 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:44.510348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c72f7f-2103-414f-8ce5-70a08a5d3690 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:44.516318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d582d40-c52a-4ea7-a1a6-303c4f3e7d88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:44.516926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d582d40-c52a-4ea7-a1a6-303c4f3e7d88 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:44.517200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d582d40-c52a-4ea7-a1a6-303c4f3e7d88 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:44.517466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d582d40-c52a-4ea7-a1a6-303c4f3e7d88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:44.526053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d582d40-c52a-4ea7-a1a6-303c4f3e7d88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:44.527618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d582d40-c52a-4ea7-a1a6-303c4f3e7d88 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:44.537330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 766/1527] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:48.319954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f41588-52d6-43c4-a5c4-33ed2d8abe89 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:48.320517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f41588-52d6-43c4-a5c4-33ed2d8abe89 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:48.320517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f41588-52d6-43c4-a5c4-33ed2d8abe89 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:48.321982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:34:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:34:48.328300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f61455-e79c-460d-ba45-facd5a43afd2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:48.328813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f61455-e79c-460d-ba45-facd5a43afd2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:48.328914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f61455-e79c-460d-ba45-facd5a43afd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:48.350405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-11f61455-e79c-460d-ba45-facd5a43afd2 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:48.351628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-11f61455-e79c-460d-ba45-facd5a43afd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:48.360186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:34:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:48.653837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f41588-52d6-43c4-a5c4-33ed2d8abe89 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:48.655948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-61f41588-52d6-43c4-a5c4-33ed2d8abe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:48.736472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f41588-52d6-43c4-a5c4-33ed2d8abe89 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:48.740125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c8c94-478a-4d89-9306-d88136223f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:48.740726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c8c94-478a-4d89-9306-d88136223f15 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:48.741013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c8c94-478a-4d89-9306-d88136223f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:48.741281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c8c94-478a-4d89-9306-d88136223f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:48.751262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c8c94-478a-4d89-9306-d88136223f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:48.752714 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c8c94-478a-4d89-9306-d88136223f15 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:48.762908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 763/1530] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:50.695680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32408234-eddb-42ec-a52e-db34c9757552 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:50.696244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32408234-eddb-42ec-a52e-db34c9757552 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:50.696244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32408234-eddb-42ec-a52e-db34c9757552 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:50.698478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:34:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:34:50.704044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bc112-f2ab-4901-8bf4-3f6525cb1ad3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:50.704658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bc112-f2ab-4901-8bf4-3f6525cb1ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:50.704658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bc112-f2ab-4901-8bf4-3f6525cb1ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:50.724855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cd4bc112-f2ab-4901-8bf4-3f6525cb1ad3 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:50.726074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cd4bc112-f2ab-4901-8bf4-3f6525cb1ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:50.734754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:34:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:51.022391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32408234-eddb-42ec-a52e-db34c9757552 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:51.034337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32408234-eddb-42ec-a52e-db34c9757552 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:51.039231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c799754-cda0-4c26-9665-7daebab11164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:51.040195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799754-cda0-4c26-9665-7daebab11164 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:51.040511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799754-cda0-4c26-9665-7daebab11164 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:51.040881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799754-cda0-4c26-9665-7daebab11164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:51.052138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c799754-cda0-4c26-9665-7daebab11164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:51.054168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c799754-cda0-4c26-9665-7daebab11164 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:51.069464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 769/1533] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:34:54.602395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a63821-8e9f-4731-b25a-f2e5287da866 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:54.602941 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a63821-8e9f-4731-b25a-f2e5287da866 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:54.602941 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a63821-8e9f-4731-b25a-f2e5287da866 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:54.604633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:34:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:34:54.609929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cfda8-58ae-4f13-a8b4-f4221f617750 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:54.610401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cfda8-58ae-4f13-a8b4-f4221f617750 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:54.610539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cfda8-58ae-4f13-a8b4-f4221f617750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:54.631296 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2c4cfda8-58ae-4f13-a8b4-f4221f617750 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:54.632508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2c4cfda8-58ae-4f13-a8b4-f4221f617750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:54.641525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:34:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:34:55.202167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a63821-8e9f-4731-b25a-f2e5287da866 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:55.203878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-14a63821-8e9f-4731-b25a-f2e5287da866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:55.271415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:34:55.286028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a63821-8e9f-4731-b25a-f2e5287da866 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:55.289719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21648ea3-29a8-490f-961a-920e131cef41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:55.290326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648ea3-29a8-490f-961a-920e131cef41 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:55.290611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648ea3-29a8-490f-961a-920e131cef41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:55.290880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648ea3-29a8-490f-961a-920e131cef41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:55.300090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21648ea3-29a8-490f-961a-920e131cef41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:55.301522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21648ea3-29a8-490f-961a-920e131cef41 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:55.311700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 766/1536] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:34:57.280877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e6950-da24-4087-be17-a3abf476a23a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:57.281481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e6950-da24-4087-be17-a3abf476a23a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:57.281481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e6950-da24-4087-be17-a3abf476a23a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:57.282921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:34:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:34:57.288468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30041a1a-d44c-47f9-b52d-803653e6b293 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:57.288863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30041a1a-d44c-47f9-b52d-803653e6b293 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:57.289022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30041a1a-d44c-47f9-b52d-803653e6b293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:57.310293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-30041a1a-d44c-47f9-b52d-803653e6b293 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:34:57.311497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-30041a1a-d44c-47f9-b52d-803653e6b293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:57.319424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:34:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:34:57.880504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e6950-da24-4087-be17-a3abf476a23a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:34:57.882241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-533e6950-da24-4087-be17-a3abf476a23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:34:57.896047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e6950-da24-4087-be17-a3abf476a23a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:34:57.899597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0a333-e69f-4cb2-8915-d25ee1d32ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:34:57.900195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0a333-e69f-4cb2-8915-d25ee1d32ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:34:57.900533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0a333-e69f-4cb2-8915-d25ee1d32ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:34:57.901292 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0a333-e69f-4cb2-8915-d25ee1d32ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:34:57.910651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0a333-e69f-4cb2-8915-d25ee1d32ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:34:57.912008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0a333-e69f-4cb2-8915-d25ee1d32ec6 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:34:57.921518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 772/1539] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:34:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:35:07.303526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=11 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:35:07.917504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:25.073418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f763805-def3-48b0-bb30-8837c37e0a31 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.075147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f763805-def3-48b0-bb30-8837c37e0a31 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.075652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f763805-def3-48b0-bb30-8837c37e0a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.102836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8f763805-def3-48b0-bb30-8837c37e0a31 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.104273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8f763805-def3-48b0-bb30-8837c37e0a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.121059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 768/1540] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:36:25.130345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dbf63-97ad-482b-a253-d86aae4d556f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.130979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dbf63-97ad-482b-a253-d86aae4d556f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.130979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dbf63-97ad-482b-a253-d86aae4d556f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.170849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f52dbf63-97ad-482b-a253-d86aae4d556f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.172484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f52dbf63-97ad-482b-a253-d86aae4d556f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.187582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 773/1541] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:36:25.192125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f763805-def3-48b0-bb30-8837c37e0a31 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.194015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83026a-8788-462b-817d-59d971dedca8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.194542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83026a-8788-462b-817d-59d971dedca8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.195116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83026a-8788-462b-817d-59d971dedca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.209180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f763805-def3-48b0-bb30-8837c37e0a31 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.222509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee02cbd-ceed-4287-8634-236c8db8723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.222763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee02cbd-ceed-4287-8634-236c8db8723e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.222867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee02cbd-ceed-4287-8634-236c8db8723e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.223116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee02cbd-ceed-4287-8634-236c8db8723e admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.223794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee02cbd-ceed-4287-8634-236c8db8723e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:25.224134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee02cbd-ceed-4287-8634-236c8db8723e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.224755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4b83026a-8788-462b-817d-59d971dedca8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.226164 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee02cbd-ceed-4287-8634-236c8db8723e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.226485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4b83026a-8788-462b-817d-59d971dedca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.246213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 769/1542] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:25.247707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12dfc6-cd65-4d22-9f55-27db2ac3c158 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.247999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12dfc6-cd65-4d22-9f55-27db2ac3c158 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.248275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12dfc6-cd65-4d22-9f55-27db2ac3c158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.250239 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 774/1543] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.255177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83026a-8788-462b-817d-59d971dedca8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.266627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83026a-8788-462b-817d-59d971dedca8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.279618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6a12dfc6-cd65-4d22-9f55-27db2ac3c158 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.281085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6a12dfc6-cd65-4d22-9f55-27db2ac3c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.281987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4efed-550c-4839-8b3c-13674a2ead29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.282678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4efed-550c-4839-8b3c-13674a2ead29 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.282961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4efed-550c-4839-8b3c-13674a2ead29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.283329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4efed-550c-4839-8b3c-13674a2ead29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.283976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4efed-550c-4839-8b3c-13674a2ead29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.286202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4efed-550c-4839-8b3c-13674a2ead29 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.313614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 770/1544] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.313614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9893b6f-9a37-4258-b060-4dcd1d28c9d6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.313614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9893b6f-9a37-4258-b060-4dcd1d28c9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.314008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9893b6f-9a37-4258-b060-4dcd1d28c9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.353663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e9893b6f-9a37-4258-b060-4dcd1d28c9d6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.359287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 775/1545] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/projects => generated 374 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:36:25.361232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338195b-c825-42c5-b5a1-81ccb6e16f3a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.361232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338195b-c825-42c5-b5a1-81ccb6e16f3a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.361232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338195b-c825-42c5-b5a1-81ccb6e16f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.365917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e9893b6f-9a37-4258-b060-4dcd1d28c9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.400483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1338195b-c825-42c5-b5a1-81ccb6e16f3a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.410419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1338195b-c825-42c5-b5a1-81ccb6e16f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.446303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 771/1546] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.449166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30380d-629b-404b-ba65-52b70485083d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.451241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30380d-629b-404b-ba65-52b70485083d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.452112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30380d-629b-404b-ba65-52b70485083d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.478852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7c30380d-629b-404b-ba65-52b70485083d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.481364 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7c30380d-629b-404b-ba65-52b70485083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.496322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 772/1547] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.497375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30380d-629b-404b-ba65-52b70485083d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.507329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30380d-629b-404b-ba65-52b70485083d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.508824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 776/1548] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.510140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cd5af-ba4a-4512-acfd-f86d4d970903 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.510422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cd5af-ba4a-4512-acfd-f86d4d970903 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.510686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cd5af-ba4a-4512-acfd-f86d4d970903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.513106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9eb010-2520-42b2-81ba-e4ac0c036b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.513779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eb010-2520-42b2-81ba-e4ac0c036b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.514065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eb010-2520-42b2-81ba-e4ac0c036b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.514339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eb010-2520-42b2-81ba-e4ac0c036b72 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.514970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9eb010-2520-42b2-81ba-e4ac0c036b72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.517062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9eb010-2520-42b2-81ba-e4ac0c036b72 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.532216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-996cd5af-ba4a-4512-acfd-f86d4d970903 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.533564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-996cd5af-ba4a-4512-acfd-f86d4d970903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.539994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 773/1549] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:25.541231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424eaec-b499-4ead-a95d-81ce468b1b79 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.541523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424eaec-b499-4ead-a95d-81ce468b1b79 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.541835 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424eaec-b499-4ead-a95d-81ce468b1b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.549802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 777/1550] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.550826 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cd5af-ba4a-4512-acfd-f86d4d970903 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.561915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cd5af-ba4a-4512-acfd-f86d4d970903 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.563017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3424eaec-b499-4ead-a95d-81ce468b1b79 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.564223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3424eaec-b499-4ead-a95d-81ce468b1b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.566707 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18086208-b823-46ff-9c97-6f7cf101616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.567319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18086208-b823-46ff-9c97-6f7cf101616c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.567599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18086208-b823-46ff-9c97-6f7cf101616c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.567871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18086208-b823-46ff-9c97-6f7cf101616c admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.568519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18086208-b823-46ff-9c97-6f7cf101616c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:25.568822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18086208-b823-46ff-9c97-6f7cf101616c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.569850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-18086208-b823-46ff-9c97-6f7cf101616c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:36:25.569850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Sep 01 17:36:25.569850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: Sep 01 17:36:25.570735 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18086208-b823-46ff-9c97-6f7cf101616c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.572856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 774/1551] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.574092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bce3db-bb12-47c3-bfd3-f32fd1274362 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.574376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bce3db-bb12-47c3-bfd3-f32fd1274362 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.574805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bce3db-bb12-47c3-bfd3-f32fd1274362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.589526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 778/1552] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:25.591196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74089920-e11e-485b-82a5-a0a21c954010 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.591332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74089920-e11e-485b-82a5-a0a21c954010 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.591543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74089920-e11e-485b-82a5-a0a21c954010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.595897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b4bce3db-bb12-47c3-bfd3-f32fd1274362 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.597198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b4bce3db-bb12-47c3-bfd3-f32fd1274362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.606485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 775/1553] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.607649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a11f51-0a46-45b9-81ae-a17c1ae443b2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.607936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a11f51-0a46-45b9-81ae-a17c1ae443b2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.608210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a11f51-0a46-45b9-81ae-a17c1ae443b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.615948 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-74089920-e11e-485b-82a5-a0a21c954010 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.617248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-74089920-e11e-485b-82a5-a0a21c954010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.626233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 779/1554] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.627326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74089920-e11e-485b-82a5-a0a21c954010 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.628051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-89a11f51-0a46-45b9-81ae-a17c1ae443b2 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.629254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-89a11f51-0a46-45b9-81ae-a17c1ae443b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.638462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74089920-e11e-485b-82a5-a0a21c954010 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.641106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 776/1555] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.642140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a11f51-0a46-45b9-81ae-a17c1ae443b2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.646168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e11c83-2ef8-447d-a3aa-6af073ce2694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.646788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e11c83-2ef8-447d-a3aa-6af073ce2694 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.647069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e11c83-2ef8-447d-a3aa-6af073ce2694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.647340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e11c83-2ef8-447d-a3aa-6af073ce2694 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.647986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e11c83-2ef8-447d-a3aa-6af073ce2694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:25.648285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e11c83-2ef8-447d-a3aa-6af073ce2694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.649930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e11c83-2ef8-447d-a3aa-6af073ce2694 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.652266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a11f51-0a46-45b9-81ae-a17c1ae443b2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.656924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391136d7-3373-4615-903b-6a9c528bc6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.657547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391136d7-3373-4615-903b-6a9c528bc6fd admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.657888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391136d7-3373-4615-903b-6a9c528bc6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.658158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391136d7-3373-4615-903b-6a9c528bc6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.658810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391136d7-3373-4615-903b-6a9c528bc6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.660411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391136d7-3373-4615-903b-6a9c528bc6fd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.666612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 780/1556] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:25.668778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893d98d-3941-4fd4-a9d9-ee2985afaef0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.669064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893d98d-3941-4fd4-a9d9-ee2985afaef0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.669336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893d98d-3941-4fd4-a9d9-ee2985afaef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.672507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 777/1557] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:25.673795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2001c-9001-40ad-a5d3-5b38abeed05d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.674320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2001c-9001-40ad-a5d3-5b38abeed05d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.674607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2001c-9001-40ad-a5d3-5b38abeed05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.691847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c893d98d-3941-4fd4-a9d9-ee2985afaef0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.692975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c893d98d-3941-4fd4-a9d9-ee2985afaef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.694328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c6a2001c-9001-40ad-a5d3-5b38abeed05d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.695497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c6a2001c-9001-40ad-a5d3-5b38abeed05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.701609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 781/1558] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.702374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893d98d-3941-4fd4-a9d9-ee2985afaef0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.707180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 778/1559] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.707180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e2f18-d351-43eb-bab4-7c1f5a0c78b6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.707180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e2f18-d351-43eb-bab4-7c1f5a0c78b6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.707180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e2f18-d351-43eb-bab4-7c1f5a0c78b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.711913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893d98d-3941-4fd4-a9d9-ee2985afaef0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.717258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfb1d8-97d9-4483-bba1-67d43c5c4353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.717892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfb1d8-97d9-4483-bba1-67d43c5c4353 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.718176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfb1d8-97d9-4483-bba1-67d43c5c4353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.718447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfb1d8-97d9-4483-bba1-67d43c5c4353 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.719070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfb1d8-97d9-4483-bba1-67d43c5c4353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.720994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfb1d8-97d9-4483-bba1-67d43c5c4353 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.725362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-268e2f18-d351-43eb-bab4-7c1f5a0c78b6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.726604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-268e2f18-d351-43eb-bab4-7c1f5a0c78b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.735416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 779/1560] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.736629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8bb03-c07d-4937-a287-13796a07cafc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.736918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8bb03-c07d-4937-a287-13796a07cafc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.737184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8bb03-c07d-4937-a287-13796a07cafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.753680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 782/1561] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/projects => generated 376 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:25.753680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfb1d8-97d9-4483-bba1-67d43c5c4353 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.757247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f3f8bb03-c07d-4937-a287-13796a07cafc None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:25.758482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f3f8bb03-c07d-4937-a287-13796a07cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:25.802573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:25.817458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfb1d8-97d9-4483-bba1-67d43c5c4353 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.821756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 780/1562] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:25.822326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55979e4-1a43-4dd1-aeb5-daf5804c9d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.822595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8bb03-c07d-4937-a287-13796a07cafc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.822992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55979e4-1a43-4dd1-aeb5-daf5804c9d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.823320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55979e4-1a43-4dd1-aeb5-daf5804c9d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.823651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55979e4-1a43-4dd1-aeb5-daf5804c9d5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.824308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55979e4-1a43-4dd1-aeb5-daf5804c9d5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:25.824620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55979e4-1a43-4dd1-aeb5-daf5804c9d5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.826274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55979e4-1a43-4dd1-aeb5-daf5804c9d5d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.832047 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8bb03-c07d-4937-a287-13796a07cafc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.837043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869772bd-253a-4932-8a08-b6448c7ae5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.837672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869772bd-253a-4932-8a08-b6448c7ae5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.837957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869772bd-253a-4932-8a08-b6448c7ae5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.838226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869772bd-253a-4932-8a08-b6448c7ae5e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.838336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 783/1563] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:25.839563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869772bd-253a-4932-8a08-b6448c7ae5e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.839778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55979e4-1a43-4dd1-aeb5-daf5804c9d5d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.841022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-869772bd-253a-4932-8a08-b6448c7ae5e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:36:25.841022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Sep 01 17:36:25.841022 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: Sep 01 17:36:25.842338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869772bd-253a-4932-8a08-b6448c7ae5e3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.847251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:25.851728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55979e4-1a43-4dd1-aeb5-daf5804c9d5d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.857925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd1274-0ae1-4c5e-80ac-9f0563672f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.858469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd1274-0ae1-4c5e-80ac-9f0563672f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.858749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd1274-0ae1-4c5e-80ac-9f0563672f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.859037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd1274-0ae1-4c5e-80ac-9f0563672f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.859659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd1274-0ae1-4c5e-80ac-9f0563672f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.861249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd1274-0ae1-4c5e-80ac-9f0563672f8c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.872228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 781/1564] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/projects => generated 382 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:25.875527 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 784/1565] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:25.875670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869772bd-253a-4932-8a08-b6448c7ae5e3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.875758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd1274-0ae1-4c5e-80ac-9f0563672f8c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.884117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd1274-0ae1-4c5e-80ac-9f0563672f8c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.936904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869772bd-253a-4932-8a08-b6448c7ae5e3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.941356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5087-537a-4066-a6aa-8422539ed1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.941542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dab6e4-95f1-42fe-9890-4ca14c8b4e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.941999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5087-537a-4066-a6aa-8422539ed1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.942143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dab6e4-95f1-42fe-9890-4ca14c8b4e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.942257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5087-537a-4066-a6aa-8422539ed1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.942402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dab6e4-95f1-42fe-9890-4ca14c8b4e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.942518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5087-537a-4066-a6aa-8422539ed1a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.942682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dab6e4-95f1-42fe-9890-4ca14c8b4e8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.943149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff5087-537a-4066-a6aa-8422539ed1a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:25.943321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dab6e4-95f1-42fe-9890-4ca14c8b4e8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:25.943431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff5087-537a-4066-a6aa-8422539ed1a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.943624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dab6e4-95f1-42fe-9890-4ca14c8b4e8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.945009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff5087-537a-4066-a6aa-8422539ed1a8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.945218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dab6e4-95f1-42fe-9890-4ca14c8b4e8f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.957651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 785/1566] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:25.958529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 782/1567] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:25.958667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5087-537a-4066-a6aa-8422539ed1a8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.963044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dab6e4-95f1-42fe-9890-4ca14c8b4e8f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:25.969538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5087-537a-4066-a6aa-8422539ed1a8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.974284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b86971-557f-43dd-ae04-cd1611f5dd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.974886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b86971-557f-43dd-ae04-cd1611f5dd03 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.974990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dab6e4-95f1-42fe-9890-4ca14c8b4e8f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:25.975174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b86971-557f-43dd-ae04-cd1611f5dd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.975448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b86971-557f-43dd-ae04-cd1611f5dd03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.976096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b86971-557f-43dd-ae04-cd1611f5dd03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:25.976395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b86971-557f-43dd-ae04-cd1611f5dd03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.978017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b86971-557f-43dd-ae04-cd1611f5dd03 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.980154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7b64a-5d89-44bd-92d0-0cc5b0ebb10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:25.980756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7b64a-5d89-44bd-92d0-0cc5b0ebb10f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:25.981032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7b64a-5d89-44bd-92d0-0cc5b0ebb10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:25.981405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7b64a-5d89-44bd-92d0-0cc5b0ebb10f admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:25.982095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7b64a-5d89-44bd-92d0-0cc5b0ebb10f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:25.984005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7b64a-5d89-44bd-92d0-0cc5b0ebb10f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:25.990568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 786/1568] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:25.991243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b86971-557f-43dd-ae04-cd1611f5dd03 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.007271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 783/1569] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:25 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.008105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7b64a-5d89-44bd-92d0-0cc5b0ebb10f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.008539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b86971-557f-43dd-ae04-cd1611f5dd03 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.014846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edb6c9-2203-49ce-8ee9-2c97c452183e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.016184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edb6c9-2203-49ce-8ee9-2c97c452183e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.016540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edb6c9-2203-49ce-8ee9-2c97c452183e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.016910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edb6c9-2203-49ce-8ee9-2c97c452183e admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.017664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edb6c9-2203-49ce-8ee9-2c97c452183e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:26.017970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edb6c9-2203-49ce-8ee9-2c97c452183e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.019600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edb6c9-2203-49ce-8ee9-2c97c452183e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.019724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7b64a-5d89-44bd-92d0-0cc5b0ebb10f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.024401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cac3d-7dd3-4c06-a4d6-5c5d3a573fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.025005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cac3d-7dd3-4c06-a4d6-5c5d3a573fcc admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.025283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cac3d-7dd3-4c06-a4d6-5c5d3a573fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.025558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cac3d-7dd3-4c06-a4d6-5c5d3a573fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.026216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cac3d-7dd3-4c06-a4d6-5c5d3a573fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.028831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cac3d-7dd3-4c06-a4d6-5c5d3a573fcc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.039637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 787/1570] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:26.039637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edb6c9-2203-49ce-8ee9-2c97c452183e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.043613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edb6c9-2203-49ce-8ee9-2c97c452183e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.048237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df30a21-1b85-4a2f-a482-c27a7d08fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.048828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df30a21-1b85-4a2f-a482-c27a7d08fd01 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.049110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df30a21-1b85-4a2f-a482-c27a7d08fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.049378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df30a21-1b85-4a2f-a482-c27a7d08fd01 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.050061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df30a21-1b85-4a2f-a482-c27a7d08fd01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.051069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-6df30a21-1b85-4a2f-a482-c27a7d08fd01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:36:26.051069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Sep 01 17:36:26.051069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: Sep 01 17:36:26.052266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df30a21-1b85-4a2f-a482-c27a7d08fd01 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.066601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 784/1571] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/projects => generated 380 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.067428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cac3d-7dd3-4c06-a4d6-5c5d3a573fcc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.077899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 788/1572] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.077899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df30a21-1b85-4a2f-a482-c27a7d08fd01 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.085690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df30a21-1b85-4a2f-a482-c27a7d08fd01 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.135212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cac3d-7dd3-4c06-a4d6-5c5d3a573fcc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.140662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc71d1-e283-4ca5-9ce6-2d62aa3252b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.141333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc71d1-e283-4ca5-9ce6-2d62aa3252b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.141636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc71d1-e283-4ca5-9ce6-2d62aa3252b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.141919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc71d1-e283-4ca5-9ce6-2d62aa3252b1 admin admin] PATH_INFO: `/v3/projects/f67baaa5ea164be2b89d3289fc9225fa/users/e7617a3afd7949daaa7763a784693ca2/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.148929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2486347-fb94-4972-a68b-a9daa332608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.149830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2486347-fb94-4972-a68b-a9daa332608e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.150208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2486347-fb94-4972-a68b-a9daa332608e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.150586 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2486347-fb94-4972-a68b-a9daa332608e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.151276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2486347-fb94-4972-a68b-a9daa332608e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.154112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2486347-fb94-4972-a68b-a9daa332608e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.158304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc71d1-e283-4ca5-9ce6-2d62aa3252b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67baaa5ea164be2b89d3289fc9225fa, user_id=e7617a3afd7949daaa7763a784693ca2, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.159379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-43bc71d1-e283-4ca5-9ce6-2d62aa3252b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:36:26.159379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Sep 01 17:36:26.159379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: Sep 01 17:36:26.161617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc71d1-e283-4ca5-9ce6-2d62aa3252b1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.175531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 785/1573] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:26 2022] PUT /identity/v3/projects/f67baaa5ea164be2b89d3289fc9225fa/users/e7617a3afd7949daaa7763a784693ca2/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:26.176388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc71d1-e283-4ca5-9ce6-2d62aa3252b1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.186080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 789/1574] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/projects => generated 356 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.186080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2486347-fb94-4972-a68b-a9daa332608e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.193466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:26.209975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:26.256808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc71d1-e283-4ca5-9ce6-2d62aa3252b1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.275223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2486347-fb94-4972-a68b-a9daa332608e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.280933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02333428-37cd-41b3-90bf-22b35a63a55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.281558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02333428-37cd-41b3-90bf-22b35a63a55e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.281872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02333428-37cd-41b3-90bf-22b35a63a55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.282149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02333428-37cd-41b3-90bf-22b35a63a55e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.282842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02333428-37cd-41b3-90bf-22b35a63a55e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.284784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02333428-37cd-41b3-90bf-22b35a63a55e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.315406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 790/1575] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/projects => generated 394 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.316237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02333428-37cd-41b3-90bf-22b35a63a55e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.329768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f30d07-63a2-4ec6-85a4-fbe9113548ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.330393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f30d07-63a2-4ec6-85a4-fbe9113548ce admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.330785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f30d07-63a2-4ec6-85a4-fbe9113548ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.331068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f30d07-63a2-4ec6-85a4-fbe9113548ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.331734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f30d07-63a2-4ec6-85a4-fbe9113548ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.333688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f30d07-63a2-4ec6-85a4-fbe9113548ce admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.365785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 786/1576] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/projects => generated 374 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.365785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f30d07-63a2-4ec6-85a4-fbe9113548ce admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.385414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02333428-37cd-41b3-90bf-22b35a63a55e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.429961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f30d07-63a2-4ec6-85a4-fbe9113548ce admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.435807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cb42f-88f5-4784-b3a2-af907609a491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.436419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cb42f-88f5-4784-b3a2-af907609a491 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.436695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cb42f-88f5-4784-b3a2-af907609a491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.436963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cb42f-88f5-4784-b3a2-af907609a491 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.437598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cb42f-88f5-4784-b3a2-af907609a491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.439532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cb42f-88f5-4784-b3a2-af907609a491 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.443430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d3675-6eea-41ec-8007-71b661fd547d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.444036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d3675-6eea-41ec-8007-71b661fd547d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.444318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d3675-6eea-41ec-8007-71b661fd547d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.444600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d3675-6eea-41ec-8007-71b661fd547d admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.445283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d3675-6eea-41ec-8007-71b661fd547d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.446865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d3675-6eea-41ec-8007-71b661fd547d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.460811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 791/1577] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:26.460811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d3675-6eea-41ec-8007-71b661fd547d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.470139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d3675-6eea-41ec-8007-71b661fd547d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.474081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 787/1578] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/projects => generated 390 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.474081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cb42f-88f5-4784-b3a2-af907609a491 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.514514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:26.523431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:26.543678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca8425-affc-4d3d-b49b-f41790c85262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.544288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca8425-affc-4d3d-b49b-f41790c85262 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.544638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca8425-affc-4d3d-b49b-f41790c85262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.544921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca8425-affc-4d3d-b49b-f41790c85262 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.545248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cb42f-88f5-4784-b3a2-af907609a491 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.545544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca8425-affc-4d3d-b49b-f41790c85262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.547536 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca8425-affc-4d3d-b49b-f41790c85262 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.549879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ce2c2-e5b8-4f8e-a535-299b84e85932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.550481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ce2c2-e5b8-4f8e-a535-299b84e85932 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.550756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ce2c2-e5b8-4f8e-a535-299b84e85932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.551025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ce2c2-e5b8-4f8e-a535-299b84e85932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.551661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ce2c2-e5b8-4f8e-a535-299b84e85932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.553552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ce2c2-e5b8-4f8e-a535-299b84e85932 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.566112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 788/1579] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:26.566947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ce2c2-e5b8-4f8e-a535-299b84e85932 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.573153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 792/1580] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.575473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4dae9-7088-43ea-b6c1-16da12145bf2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.575755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4dae9-7088-43ea-b6c1-16da12145bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.576039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4dae9-7088-43ea-b6c1-16da12145bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.578246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ce2c2-e5b8-4f8e-a535-299b84e85932 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.583822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9c86e-d587-4441-ba29-1344ae30d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.584432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9c86e-d587-4441-ba29-1344ae30d36c admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.584720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9c86e-d587-4441-ba29-1344ae30d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.585253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9c86e-d587-4441-ba29-1344ae30d36c admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.585893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9c86e-d587-4441-ba29-1344ae30d36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.587846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9c86e-d587-4441-ba29-1344ae30d36c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.609759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 789/1581] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/users => generated 279 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.610715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9c86e-d587-4441-ba29-1344ae30d36c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.613170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-71c4dae9-7088-43ea-b6c1-16da12145bf2 None None] MFA Rules not processed for user `e7617a3afd7949daaa7763a784693ca2`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:26.614423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-71c4dae9-7088-43ea-b6c1-16da12145bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:26.621069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9c86e-d587-4441-ba29-1344ae30d36c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.625923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6e974-6710-432d-9467-21c3143befd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.626520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6e974-6710-432d-9467-21c3143befd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.626801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6e974-6710-432d-9467-21c3143befd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.627076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6e974-6710-432d-9467-21c3143befd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.627704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6e974-6710-432d-9467-21c3143befd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.629628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6e974-6710-432d-9467-21c3143befd4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.656721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 790/1582] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/users => generated 298 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.656721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6e974-6710-432d-9467-21c3143befd4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.665370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6e974-6710-432d-9467-21c3143befd4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.671023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf07cfa-5db5-4103-9664-d6262b9f794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.671631 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf07cfa-5db5-4103-9664-d6262b9f794d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.671910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf07cfa-5db5-4103-9664-d6262b9f794d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.672333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf07cfa-5db5-4103-9664-d6262b9f794d admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.672972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf07cfa-5db5-4103-9664-d6262b9f794d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.674926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf07cfa-5db5-4103-9664-d6262b9f794d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.693107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 793/1583] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:26.694024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4dae9-7088-43ea-b6c1-16da12145bf2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.702889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 791/1584] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/users => generated 288 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.705372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4dae9-7088-43ea-b6c1-16da12145bf2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.706640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf07cfa-5db5-4103-9664-d6262b9f794d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.709997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ac88a-8f97-4327-8c4e-f0e0cd12d609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.710608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ac88a-8f97-4327-8c4e-f0e0cd12d609 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.710891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ac88a-8f97-4327-8c4e-f0e0cd12d609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.711168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ac88a-8f97-4327-8c4e-f0e0cd12d609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.711813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ac88a-8f97-4327-8c4e-f0e0cd12d609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.713551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ac88a-8f97-4327-8c4e-f0e0cd12d609 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.717844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf07cfa-5db5-4103-9664-d6262b9f794d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.723154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b80631-bffb-44bd-99e2-205e796fce3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.723820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b80631-bffb-44bd-99e2-205e796fce3f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.724293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b80631-bffb-44bd-99e2-205e796fce3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.724620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b80631-bffb-44bd-99e2-205e796fce3f admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.728368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 794/1585] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:26.728510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b80631-bffb-44bd-99e2-205e796fce3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.728628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ac88a-8f97-4327-8c4e-f0e0cd12d609 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.728733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b80631-bffb-44bd-99e2-205e796fce3f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.739891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ac88a-8f97-4327-8c4e-f0e0cd12d609 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.746499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f3741-17dc-44b4-b42a-b997486301f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.747119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f3741-17dc-44b4-b42a-b997486301f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.747400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f3741-17dc-44b4-b42a-b997486301f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.747675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f3741-17dc-44b4-b42a-b997486301f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.748317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f3741-17dc-44b4-b42a-b997486301f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.749953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f3741-17dc-44b4-b42a-b997486301f7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.760959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 795/1586] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:26.761720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f3741-17dc-44b4-b42a-b997486301f7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.763181 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 792/1587] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:26 2022] POST /identity/v3/users => generated 296 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:26.764768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b80631-bffb-44bd-99e2-205e796fce3f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.772192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f3741-17dc-44b4-b42a-b997486301f7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.777046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b80631-bffb-44bd-99e2-205e796fce3f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.777226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505238f-db18-446c-8cce-546611a0d89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.777811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505238f-db18-446c-8cce-546611a0d89a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.778088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505238f-db18-446c-8cce-546611a0d89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.778360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505238f-db18-446c-8cce-546611a0d89a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.779000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505238f-db18-446c-8cce-546611a0d89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.780609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505238f-db18-446c-8cce-546611a0d89a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.782385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b52a33-465a-4630-a24d-90a0ae72c792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.783075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b52a33-465a-4630-a24d-90a0ae72c792 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.783354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b52a33-465a-4630-a24d-90a0ae72c792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.783631 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b52a33-465a-4630-a24d-90a0ae72c792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.784281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b52a33-465a-4630-a24d-90a0ae72c792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.786255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b52a33-465a-4630-a24d-90a0ae72c792 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.793319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 796/1588] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:26.794472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505238f-db18-446c-8cce-546611a0d89a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.799983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 793/1589] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:26.801171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b52a33-465a-4630-a24d-90a0ae72c792 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.806470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505238f-db18-446c-8cce-546611a0d89a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.811979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09509320-9a54-4be8-8da0-600f6d057265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.812595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09509320-9a54-4be8-8da0-600f6d057265 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.812886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09509320-9a54-4be8-8da0-600f6d057265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.813155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09509320-9a54-4be8-8da0-600f6d057265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.813824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09509320-9a54-4be8-8da0-600f6d057265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.815448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09509320-9a54-4be8-8da0-600f6d057265 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.816352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b52a33-465a-4630-a24d-90a0ae72c792 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.822250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034be74-fa5c-4fbe-9230-5442f6161c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.822849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034be74-fa5c-4fbe-9230-5442f6161c1d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.823154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034be74-fa5c-4fbe-9230-5442f6161c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.824488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034be74-fa5c-4fbe-9230-5442f6161c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.824488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034be74-fa5c-4fbe-9230-5442f6161c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.825814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034be74-fa5c-4fbe-9230-5442f6161c1d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.830643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 797/1590] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:26.830643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430967bb-e05c-4c16-84a6-99194260e407 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.830643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430967bb-e05c-4c16-84a6-99194260e407 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.830643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430967bb-e05c-4c16-84a6-99194260e407 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.830643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 798/1591] 10.222.0.100 () {54 vars in 967 bytes} [Thu Sep 1 17:36:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:26.830643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430967bb-e05c-4c16-84a6-99194260e407 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.843875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 794/1592] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:26.843875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034be74-fa5c-4fbe-9230-5442f6161c1d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.844435 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430967bb-e05c-4c16-84a6-99194260e407 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.848521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e99a69-9854-43ef-9e0f-ee555ee6effa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.849269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e99a69-9854-43ef-9e0f-ee555ee6effa admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.849482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e99a69-9854-43ef-9e0f-ee555ee6effa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.849809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e99a69-9854-43ef-9e0f-ee555ee6effa admin admin] PATH_INFO: `/v3/projects/49ce77df03d34465a8d809e9c82fb42f/users/781a9b0e524c46fc89908e1a1bc4c14c/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.852933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034be74-fa5c-4fbe-9230-5442f6161c1d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.859366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdf16d-79c8-4f25-bdc2-59fd6acd986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.859988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdf16d-79c8-4f25-bdc2-59fd6acd986a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.860265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdf16d-79c8-4f25-bdc2-59fd6acd986a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.860537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdf16d-79c8-4f25-bdc2-59fd6acd986a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.861174 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdf16d-79c8-4f25-bdc2-59fd6acd986a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.862865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdf16d-79c8-4f25-bdc2-59fd6acd986a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.866363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e99a69-9854-43ef-9e0f-ee555ee6effa admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ce77df03d34465a8d809e9c82fb42f, user_id=781a9b0e524c46fc89908e1a1bc4c14c, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.869015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e99a69-9854-43ef-9e0f-ee555ee6effa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.874594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 795/1593] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:26.875711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdf16d-79c8-4f25-bdc2-59fd6acd986a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.886235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdf16d-79c8-4f25-bdc2-59fd6acd986a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.897873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 799/1594] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:26 2022] PUT /identity/v3/projects/49ce77df03d34465a8d809e9c82fb42f/users/781a9b0e524c46fc89908e1a1bc4c14c/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:26.897873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e99a69-9854-43ef-9e0f-ee555ee6effa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:26.952661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:26.954231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd09052-187a-4656-bdf6-c7b434f8b129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.955094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd09052-187a-4656-bdf6-c7b434f8b129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.955380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd09052-187a-4656-bdf6-c7b434f8b129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.955913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd09052-187a-4656-bdf6-c7b434f8b129 admin admin] PATH_INFO: `/v3/projects/19a5df264e224d8b901f524b41268ffc/users/f8ad7f0ec67542279d17b18e16da6bde/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.973155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd09052-187a-4656-bdf6-c7b434f8b129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a5df264e224d8b901f524b41268ffc, user_id=f8ad7f0ec67542279d17b18e16da6bde, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.975928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd09052-187a-4656-bdf6-c7b434f8b129 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.976919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e99a69-9854-43ef-9e0f-ee555ee6effa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:26.983468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909a0b4-f4e5-49c8-bd83-8f17ae7ef4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:26.984292 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909a0b4-f4e5-49c8-bd83-8f17ae7ef4aa admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:26.984619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909a0b4-f4e5-49c8-bd83-8f17ae7ef4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:26.985475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909a0b4-f4e5-49c8-bd83-8f17ae7ef4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:26.986264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909a0b4-f4e5-49c8-bd83-8f17ae7ef4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:26.987998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909a0b4-f4e5-49c8-bd83-8f17ae7ef4aa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:26.999855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 796/1595] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:26 2022] PUT /identity/v3/projects/19a5df264e224d8b901f524b41268ffc/users/f8ad7f0ec67542279d17b18e16da6bde/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:27.002852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd09052-187a-4656-bdf6-c7b434f8b129 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.004672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 800/1596] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:27.004672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909a0b4-f4e5-49c8-bd83-8f17ae7ef4aa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.020240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909a0b4-f4e5-49c8-bd83-8f17ae7ef4aa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.036485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:27.085331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009ea56-418d-42ef-8c51-3f3422bb34da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.086166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ea56-418d-42ef-8c51-3f3422bb34da admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.086322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd09052-187a-4656-bdf6-c7b434f8b129 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.086621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ea56-418d-42ef-8c51-3f3422bb34da admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.086982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ea56-418d-42ef-8c51-3f3422bb34da admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.087726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009ea56-418d-42ef-8c51-3f3422bb34da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.090688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009ea56-418d-42ef-8c51-3f3422bb34da admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.094942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb1f2e-1495-4209-995b-119018f474e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.095574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb1f2e-1495-4209-995b-119018f474e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.095883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb1f2e-1495-4209-995b-119018f474e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.096149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb1f2e-1495-4209-995b-119018f474e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.096825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb1f2e-1495-4209-995b-119018f474e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.098522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb1f2e-1495-4209-995b-119018f474e7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.101864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 801/1597] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:27 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:27.103162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9f9e3-7e42-4c85-81ac-78ae8143edc2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.103450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9f9e3-7e42-4c85-81ac-78ae8143edc2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.103722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9f9e3-7e42-4c85-81ac-78ae8143edc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.111137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 797/1598] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:26 2022] GET /identity/v3/roles => generated 1427 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:27.112080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb1f2e-1495-4209-995b-119018f474e7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.127613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb1f2e-1495-4209-995b-119018f474e7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.134122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab98c8e-73bd-437d-bc6d-487a1115ec6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.134792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab98c8e-73bd-437d-bc6d-487a1115ec6a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.135077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab98c8e-73bd-437d-bc6d-487a1115ec6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.135362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab98c8e-73bd-437d-bc6d-487a1115ec6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.136671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab98c8e-73bd-437d-bc6d-487a1115ec6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.138502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab98c8e-73bd-437d-bc6d-487a1115ec6a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.149085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d3f9f9e3-7e42-4c85-81ac-78ae8143edc2 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:27.149889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 798/1599] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:27 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:27.150216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab98c8e-73bd-437d-bc6d-487a1115ec6a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.150684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d3f9f9e3-7e42-4c85-81ac-78ae8143edc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:27.160451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab98c8e-73bd-437d-bc6d-487a1115ec6a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.165419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba5580-7c03-4700-b968-47bf1e14abda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.166079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba5580-7c03-4700-b968-47bf1e14abda admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.166358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba5580-7c03-4700-b968-47bf1e14abda admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.166630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba5580-7c03-4700-b968-47bf1e14abda admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.167309 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba5580-7c03-4700-b968-47bf1e14abda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.168941 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba5580-7c03-4700-b968-47bf1e14abda admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.184688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 799/1600] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:27 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:27.184688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efcb71-ba89-4a7c-ba4c-75c90116f632 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.184688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efcb71-ba89-4a7c-ba4c-75c90116f632 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.184688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efcb71-ba89-4a7c-ba4c-75c90116f632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.217365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-32efcb71-ba89-4a7c-ba4c-75c90116f632 None None] MFA Rules not processed for user `781a9b0e524c46fc89908e1a1bc4c14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:27.218535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-32efcb71-ba89-4a7c-ba4c-75c90116f632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:27.223829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 802/1601] 10.222.0.100 () {58 vars in 1064 bytes} [Thu Sep 1 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:27.226487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9f9e3-7e42-4c85-81ac-78ae8143edc2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.237292 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9f9e3-7e42-4c85-81ac-78ae8143edc2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.241954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef0370-b2de-4899-a961-c03c7e516609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.242598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef0370-b2de-4899-a961-c03c7e516609 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.242872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef0370-b2de-4899-a961-c03c7e516609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.243142 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef0370-b2de-4899-a961-c03c7e516609 admin admin] PATH_INFO: `/v3/projects/df3925a7fb6c4d099c4f160d42870a3b/users/bcea72e4fc77431a96e7acf9d68eb083/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.258315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef0370-b2de-4899-a961-c03c7e516609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3925a7fb6c4d099c4f160d42870a3b, user_id=bcea72e4fc77431a96e7acf9d68eb083, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.261763 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef0370-b2de-4899-a961-c03c7e516609 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.274822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 803/1602] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:27 2022] PUT /identity/v3/projects/df3925a7fb6c4d099c4f160d42870a3b/users/bcea72e4fc77431a96e7acf9d68eb083/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:27.276047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08b71b-322c-4fb4-a1da-89d68239a053 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.276355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08b71b-322c-4fb4-a1da-89d68239a053 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.276629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08b71b-322c-4fb4-a1da-89d68239a053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.289947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 800/1603] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:27.290927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efcb71-ba89-4a7c-ba4c-75c90116f632 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.302117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efcb71-ba89-4a7c-ba4c-75c90116f632 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.310566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3b08b71b-322c-4fb4-a1da-89d68239a053 None None] MFA Rules not processed for user `f8ad7f0ec67542279d17b18e16da6bde`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:27.311857 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3b08b71b-322c-4fb4-a1da-89d68239a053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:27.335753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:27.377028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c5d8d-c45e-4557-b1c5-3ea89e8d7885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.377778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c5d8d-c45e-4557-b1c5-3ea89e8d7885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.378020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c5d8d-c45e-4557-b1c5-3ea89e8d7885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.378294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c5d8d-c45e-4557-b1c5-3ea89e8d7885 admin admin] PATH_INFO: `/v3/projects/b01df46ff9744c2f830459547d7e4dd6/users/5647214c5b3f438eb3ef1ddc14153d9e/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.388377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 804/1604] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:27.389328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08b71b-322c-4fb4-a1da-89d68239a053 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.394768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c5d8d-c45e-4557-b1c5-3ea89e8d7885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01df46ff9744c2f830459547d7e4dd6, user_id=5647214c5b3f438eb3ef1ddc14153d9e, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.397525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c5d8d-c45e-4557-b1c5-3ea89e8d7885 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.408915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08b71b-322c-4fb4-a1da-89d68239a053 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.411907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 801/1605] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:27 2022] PUT /identity/v3/projects/b01df46ff9744c2f830459547d7e4dd6/users/5647214c5b3f438eb3ef1ddc14153d9e/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:27.415886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c5d8d-c45e-4557-b1c5-3ea89e8d7885 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.422188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:27.485031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c5d8d-c45e-4557-b1c5-3ea89e8d7885 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.485233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca0e3d-8877-4ab8-81bb-338a39f15a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.485841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca0e3d-8877-4ab8-81bb-338a39f15a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.486127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca0e3d-8877-4ab8-81bb-338a39f15a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.486403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca0e3d-8877-4ab8-81bb-338a39f15a7c admin admin] PATH_INFO: `/v3/projects/dba6ce0b2305423790e5153ee702f1c4/users/d7667ec51e3b4e5e8c352dcbf2e08e2e/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.490200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c4bad-aec9-42cf-8cc9-cad642545770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.490866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c4bad-aec9-42cf-8cc9-cad642545770 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.491140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c4bad-aec9-42cf-8cc9-cad642545770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.491411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c4bad-aec9-42cf-8cc9-cad642545770 admin admin] PATH_INFO: `/v3/projects/843dd8b5651d4a9096effa8b264b6ace/users/341fc92dd6ef49d6b18ef1f0f70543f2/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.503301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca0e3d-8877-4ab8-81bb-338a39f15a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba6ce0b2305423790e5153ee702f1c4, user_id=d7667ec51e3b4e5e8c352dcbf2e08e2e, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.506129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca0e3d-8877-4ab8-81bb-338a39f15a7c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.510281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c4bad-aec9-42cf-8cc9-cad642545770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=843dd8b5651d4a9096effa8b264b6ace, user_id=341fc92dd6ef49d6b18ef1f0f70543f2, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.514313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c4bad-aec9-42cf-8cc9-cad642545770 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.523626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 805/1606] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:27 2022] PUT /identity/v3/projects/dba6ce0b2305423790e5153ee702f1c4/users/d7667ec51e3b4e5e8c352dcbf2e08e2e/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:27.524518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca0e3d-8877-4ab8-81bb-338a39f15a7c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.541329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 802/1607] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:27 2022] PUT /identity/v3/projects/843dd8b5651d4a9096effa8b264b6ace/users/341fc92dd6ef49d6b18ef1f0f70543f2/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:27.542536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c4bad-aec9-42cf-8cc9-cad642545770 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:27.608746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca0e3d-8877-4ab8-81bb-338a39f15a7c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.620736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c4bad-aec9-42cf-8cc9-cad642545770 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:27.626487 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cbf1b-6292-498e-b534-907107848b57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.627095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cbf1b-6292-498e-b534-907107848b57 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.627383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cbf1b-6292-498e-b534-907107848b57 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.627678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cbf1b-6292-498e-b534-907107848b57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.664133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:27.674544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080dc4-ff58-4356-85e3-6542ac86cbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:27.675211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080dc4-ff58-4356-85e3-6542ac86cbb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.675490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080dc4-ff58-4356-85e3-6542ac86cbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.675767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080dc4-ff58-4356-85e3-6542ac86cbb7 admin admin] PATH_INFO: `/v3/projects/e11b8d4a14a9460eb4ce9967f86e4757/users/bcc538bc852f404699d91d17023ae620/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.690371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080dc4-ff58-4356-85e3-6542ac86cbb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11b8d4a14a9460eb4ce9967f86e4757, user_id=bcc538bc852f404699d91d17023ae620, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.691501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cbf1b-6292-498e-b534-907107848b57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:27.693015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080dc4-ff58-4356-85e3-6542ac86cbb7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.693134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cbf1b-6292-498e-b534-907107848b57 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:27.702982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 803/1608] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:27.704285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59afee0-112a-4514-97e2-7fd90e8fc461 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.704651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59afee0-112a-4514-97e2-7fd90e8fc461 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.704934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59afee0-112a-4514-97e2-7fd90e8fc461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.708460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 806/1609] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:27 2022] PUT /identity/v3/projects/e11b8d4a14a9460eb4ce9967f86e4757/users/bcc538bc852f404699d91d17023ae620/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:27.709545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439bce8-f38d-475e-847d-cbabb02bf9e0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.710045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439bce8-f38d-475e-847d-cbabb02bf9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.710318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439bce8-f38d-475e-847d-cbabb02bf9e0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.711684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 807/1610] 10.222.0.100 () {54 vars in 967 bytes} [Thu Sep 1 17:36:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:27.712787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba6cf8-7c0c-470f-8a29-4cf490c2eea1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.713063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba6cf8-7c0c-470f-8a29-4cf490c2eea1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.713331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba6cf8-7c0c-470f-8a29-4cf490c2eea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.734480 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c59afee0-112a-4514-97e2-7fd90e8fc461 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:27.734704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2cba6cf8-7c0c-470f-8a29-4cf490c2eea1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:27.735743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2cba6cf8-7c0c-470f-8a29-4cf490c2eea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:27.735857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c59afee0-112a-4514-97e2-7fd90e8fc461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:27.753892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:27.802753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 808/1611] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:27.804851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 804/1612] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:27.806152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd63e-b585-4a50-aee1-cf7d18c1b107 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.806517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd63e-b585-4a50-aee1-cf7d18c1b107 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.806806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd63e-b585-4a50-aee1-cf7d18c1b107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.818614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52b60e-6dc9-4d0a-a38a-c43356182d89 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.818934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52b60e-6dc9-4d0a-a38a-c43356182d89 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.819169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52b60e-6dc9-4d0a-a38a-c43356182d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.839909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1cbdd63e-b585-4a50-aee1-cf7d18c1b107 None None] MFA Rules not processed for user `341fc92dd6ef49d6b18ef1f0f70543f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:27.841129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1cbdd63e-b585-4a50-aee1-cf7d18c1b107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:27.850459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0c52b60e-6dc9-4d0a-a38a-c43356182d89 None None] MFA Rules not processed for user `d7667ec51e3b4e5e8c352dcbf2e08e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:27.851643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0c52b60e-6dc9-4d0a-a38a-c43356182d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:27.913353 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 805/1613] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:27.914821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd8ef2-aefb-4e9c-9387-998b1379979f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.915234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd8ef2-aefb-4e9c-9387-998b1379979f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.915518 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd8ef2-aefb-4e9c-9387-998b1379979f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.928545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 809/1614] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:27.928545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4820c-d2db-4bd7-b0d6-4424eec25d54 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:27.928545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4820c-d2db-4bd7-b0d6-4424eec25d54 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:27.928545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4820c-d2db-4bd7-b0d6-4424eec25d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:27.937804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-87cd8ef2-aefb-4e9c-9387-998b1379979f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:27.939004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-87cd8ef2-aefb-4e9c-9387-998b1379979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:27.946955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-96a4820c-d2db-4bd7-b0d6-4424eec25d54 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:27.947422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 806/1615] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:27.948126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-96a4820c-d2db-4bd7-b0d6-4424eec25d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:27.952049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd8ef2-aefb-4e9c-9387-998b1379979f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.007421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:28.012429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd8ef2-aefb-4e9c-9387-998b1379979f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.016158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e67952-fb87-4467-8cfe-dae551b472dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.016856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e67952-fb87-4467-8cfe-dae551b472dd service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.017151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e67952-fb87-4467-8cfe-dae551b472dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.017424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e67952-fb87-4467-8cfe-dae551b472dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.021297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 810/1616] 10.222.0.100 () {58 vars in 1064 bytes} [Thu Sep 1 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:28.022158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4820c-d2db-4bd7-b0d6-4424eec25d54 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.023419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-96a4820c-d2db-4bd7-b0d6-4424eec25d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:28.027249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e67952-fb87-4467-8cfe-dae551b472dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.028300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e67952-fb87-4467-8cfe-dae551b472dd service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.035242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4820c-d2db-4bd7-b0d6-4424eec25d54 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.038376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 807/1617] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:27 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:28.039160 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e67952-fb87-4467-8cfe-dae551b472dd service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.039623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b7f5b-4b18-4776-92bc-b0833743643f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.040316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b7f5b-4b18-4776-92bc-b0833743643f service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.040597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b7f5b-4b18-4776-92bc-b0833743643f service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.040870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b7f5b-4b18-4776-92bc-b0833743643f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.048660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e67952-fb87-4467-8cfe-dae551b472dd service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.051719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b7f5b-4b18-4776-92bc-b0833743643f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.053386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b7f5b-4b18-4776-92bc-b0833743643f service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.053928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeab22-6723-41d2-9ccc-889002784297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.054526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeab22-6723-41d2-9ccc-889002784297 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.054809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeab22-6723-41d2-9ccc-889002784297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.055084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeab22-6723-41d2-9ccc-889002784297 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.055720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeab22-6723-41d2-9ccc-889002784297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.058198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeab22-6723-41d2-9ccc-889002784297 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.063860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 811/1618] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:28.064756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b7f5b-4b18-4776-92bc-b0833743643f service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.077193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b7f5b-4b18-4776-92bc-b0833743643f service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.083211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163ca75-3580-48ad-905e-0aaffe35c0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.083822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163ca75-3580-48ad-905e-0aaffe35c0bd admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.084101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163ca75-3580-48ad-905e-0aaffe35c0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.084481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163ca75-3580-48ad-905e-0aaffe35c0bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.084990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163ca75-3580-48ad-905e-0aaffe35c0bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.086928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163ca75-3580-48ad-905e-0aaffe35c0bd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.094721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 808/1619] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:28 2022] POST /identity/v3/projects => generated 372 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:28.098779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeab22-6723-41d2-9ccc-889002784297 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.124207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 812/1620] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:28 2022] POST /identity/v3/projects => generated 356 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:28.125780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b1c27-5c29-4730-9f9c-5814663f78a8 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.126071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b1c27-5c29-4730-9f9c-5814663f78a8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.126351 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b1c27-5c29-4730-9f9c-5814663f78a8 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.127998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 813/1621] 10.222.0.100 () {54 vars in 917 bytes} [Thu Sep 1 17:36:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:28.133999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b1c27-5c29-4730-9f9c-5814663f78a8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.134855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:28.173909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeab22-6723-41d2-9ccc-889002784297 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.234834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82a71c-5120-4f23-8faf-2085445ec49d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.235754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82a71c-5120-4f23-8faf-2085445ec49d service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.236075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82a71c-5120-4f23-8faf-2085445ec49d service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.236463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82a71c-5120-4f23-8faf-2085445ec49d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.242666 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b1c27-5c29-4730-9f9c-5814663f78a8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.250264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ef850-8862-47ca-bc10-741cecd3d218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.250880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ef850-8862-47ca-bc10-741cecd3d218 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.251160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ef850-8862-47ca-bc10-741cecd3d218 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.251420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ef850-8862-47ca-bc10-741cecd3d218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.327536 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ef850-8862-47ca-bc10-741cecd3d218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.329364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ef850-8862-47ca-bc10-741cecd3d218 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.340286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82a71c-5120-4f23-8faf-2085445ec49d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.342120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82a71c-5120-4f23-8faf-2085445ec49d service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.353641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 809/1622] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:28.355092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82a71c-5120-4f23-8faf-2085445ec49d service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.357816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 814/1623] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:28.359553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ef850-8862-47ca-bc10-741cecd3d218 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.366691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82a71c-5120-4f23-8faf-2085445ec49d service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.373589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919de77-de70-4a86-a2ba-95cdb1f4f2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.374225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919de77-de70-4a86-a2ba-95cdb1f4f2dd admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.374504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919de77-de70-4a86-a2ba-95cdb1f4f2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.374845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919de77-de70-4a86-a2ba-95cdb1f4f2dd admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.375468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919de77-de70-4a86-a2ba-95cdb1f4f2dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.378015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919de77-de70-4a86-a2ba-95cdb1f4f2dd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.388883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ef850-8862-47ca-bc10-741cecd3d218 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.400034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041546ff-4510-4695-82ac-66e857e2afa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.401005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041546ff-4510-4695-82ac-66e857e2afa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.402849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041546ff-4510-4695-82ac-66e857e2afa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.403601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041546ff-4510-4695-82ac-66e857e2afa3 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.404278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041546ff-4510-4695-82ac-66e857e2afa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.410811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041546ff-4510-4695-82ac-66e857e2afa3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.414267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 810/1624] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:28 2022] POST /identity/v3/users => generated 293 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:28.415043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850a6fa-45c0-4886-b599-ed572cfe48c9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.415859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850a6fa-45c0-4886-b599-ed572cfe48c9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.416185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850a6fa-45c0-4886-b599-ed572cfe48c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.446875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7850a6fa-45c0-4886-b599-ed572cfe48c9 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:28.448938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7850a6fa-45c0-4886-b599-ed572cfe48c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:28.456689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 815/1625] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:28 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:28.462028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 811/1626] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:36:28 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:28.463685 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850a6fa-45c0-4886-b599-ed572cfe48c9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.468582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041546ff-4510-4695-82ac-66e857e2afa3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.481346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850a6fa-45c0-4886-b599-ed572cfe48c9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.487925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812be7d-c7f1-497e-836a-6926d1235fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.488581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812be7d-c7f1-497e-836a-6926d1235fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.488919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812be7d-c7f1-497e-836a-6926d1235fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.489245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812be7d-c7f1-497e-836a-6926d1235fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.490899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812be7d-c7f1-497e-836a-6926d1235fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.491147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041546ff-4510-4695-82ac-66e857e2afa3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.493618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812be7d-c7f1-497e-836a-6926d1235fe8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.504362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935e5c5-95f9-4605-a9a9-4a2ed2ff804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.504991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935e5c5-95f9-4605-a9a9-4a2ed2ff804c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.505267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935e5c5-95f9-4605-a9a9-4a2ed2ff804c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.505535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935e5c5-95f9-4605-a9a9-4a2ed2ff804c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.506276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935e5c5-95f9-4605-a9a9-4a2ed2ff804c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.507888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935e5c5-95f9-4605-a9a9-4a2ed2ff804c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.511687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 812/1627] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:28 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:28.513223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812be7d-c7f1-497e-836a-6926d1235fe8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.534467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 816/1628] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:28 2022] GET /identity/v3/roles => generated 1427 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:28.534848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935e5c5-95f9-4605-a9a9-4a2ed2ff804c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.557690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935e5c5-95f9-4605-a9a9-4a2ed2ff804c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.573746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b14d9a-c76b-4f29-b49b-376e380e1074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.574716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b14d9a-c76b-4f29-b49b-376e380e1074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.574997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b14d9a-c76b-4f29-b49b-376e380e1074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.575269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b14d9a-c76b-4f29-b49b-376e380e1074 admin admin] PATH_INFO: `/v3/projects/efb42e99c1a84667a09e9cd1af4f8d4b/users/0ab7f9049c2a4511817c7e47107a9c87/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.598794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812be7d-c7f1-497e-836a-6926d1235fe8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.603143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caf64e-fb93-4218-a747-36b3f7212901 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.603764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caf64e-fb93-4218-a747-36b3f7212901 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.604045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caf64e-fb93-4218-a747-36b3f7212901 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.604318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caf64e-fb93-4218-a747-36b3f7212901 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.609085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b14d9a-c76b-4f29-b49b-376e380e1074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb42e99c1a84667a09e9cd1af4f8d4b, user_id=0ab7f9049c2a4511817c7e47107a9c87, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.612012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b14d9a-c76b-4f29-b49b-376e380e1074 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.617721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0caf64e-fb93-4218-a747-36b3f7212901 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.619339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0caf64e-fb93-4218-a747-36b3f7212901 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.627206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:28.639650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 813/1629] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:28 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:28.639893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 817/1630] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:28 2022] PUT /identity/v3/projects/efb42e99c1a84667a09e9cd1af4f8d4b/users/0ab7f9049c2a4511817c7e47107a9c87/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:28.640609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caf64e-fb93-4218-a747-36b3f7212901 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.644327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b14d9a-c76b-4f29-b49b-376e380e1074 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.659935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caf64e-fb93-4218-a747-36b3f7212901 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.669871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:28.751013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a791c-f4d3-4833-a5e5-8637c6532076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.751711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a791c-f4d3-4833-a5e5-8637c6532076 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.752005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a791c-f4d3-4833-a5e5-8637c6532076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.752702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a791c-f4d3-4833-a5e5-8637c6532076 admin admin] PATH_INFO: `/v3/projects/5602ec31052e4716845e3df6324a8d7a/users/d0a5920e58894979837e43d1974873be/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.773397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a791c-f4d3-4833-a5e5-8637c6532076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5602ec31052e4716845e3df6324a8d7a, user_id=d0a5920e58894979837e43d1974873be, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.777783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a791c-f4d3-4833-a5e5-8637c6532076 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.805623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 814/1631] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:28 2022] PUT /identity/v3/projects/5602ec31052e4716845e3df6324a8d7a/users/d0a5920e58894979837e43d1974873be/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:28.808898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a791c-f4d3-4833-a5e5-8637c6532076 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:28.833636 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b14d9a-c76b-4f29-b49b-376e380e1074 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.973782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a791c-f4d3-4833-a5e5-8637c6532076 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:28.975110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a6820-bb4f-462c-a73a-b15ee22e4eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.976448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a6820-bb4f-462c-a73a-b15ee22e4eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.977913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a6820-bb4f-462c-a73a-b15ee22e4eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.977913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a6820-bb4f-462c-a73a-b15ee22e4eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.980171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a6820-bb4f-462c-a73a-b15ee22e4eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.982231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a6820-bb4f-462c-a73a-b15ee22e4eb1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.987215 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36dd7c-e79e-48bd-a228-32f01d391706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:28.989023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36dd7c-e79e-48bd-a228-32f01d391706 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:28.989725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36dd7c-e79e-48bd-a228-32f01d391706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:28.990006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36dd7c-e79e-48bd-a228-32f01d391706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:28.991956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36dd7c-e79e-48bd-a228-32f01d391706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:28.996254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 818/1632] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:28 2022] GET /identity/v3/roles => generated 1427 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:28.996481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36dd7c-e79e-48bd-a228-32f01d391706 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:28.997310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a6820-bb4f-462c-a73a-b15ee22e4eb1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:29.015800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 815/1633] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:28 2022] GET /identity/v3/roles => generated 1427 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:29.016678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36dd7c-e79e-48bd-a228-32f01d391706 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:29.028404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36dd7c-e79e-48bd-a228-32f01d391706 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:29.034706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231716de-41ac-4707-8db1-7bcd5e869889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:29.035504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231716de-41ac-4707-8db1-7bcd5e869889 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:29.036291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231716de-41ac-4707-8db1-7bcd5e869889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:29.036745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231716de-41ac-4707-8db1-7bcd5e869889 admin admin] PATH_INFO: `/v3/projects/efb42e99c1a84667a09e9cd1af4f8d4b/users/0ab7f9049c2a4511817c7e47107a9c87/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:29.038959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231716de-41ac-4707-8db1-7bcd5e869889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb42e99c1a84667a09e9cd1af4f8d4b, user_id=0ab7f9049c2a4511817c7e47107a9c87, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:29.043226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231716de-41ac-4707-8db1-7bcd5e869889 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:29.056470 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 816/1634] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:29 2022] PUT /identity/v3/projects/efb42e99c1a84667a09e9cd1af4f8d4b/users/0ab7f9049c2a4511817c7e47107a9c87/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:29.057400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231716de-41ac-4707-8db1-7bcd5e869889 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:29.109237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a6820-bb4f-462c-a73a-b15ee22e4eb1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:29.138120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231716de-41ac-4707-8db1-7bcd5e869889 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:29.144599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56271e1f-c15c-4591-a3d9-39c16569b2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:29.145839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56271e1f-c15c-4591-a3d9-39c16569b2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:29.146106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56271e1f-c15c-4591-a3d9-39c16569b2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:29.146349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56271e1f-c15c-4591-a3d9-39c16569b2bb admin admin] PATH_INFO: `/v3/projects/5602ec31052e4716845e3df6324a8d7a/users/d0a5920e58894979837e43d1974873be/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:29.149735 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56271e1f-c15c-4591-a3d9-39c16569b2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5602ec31052e4716845e3df6324a8d7a, user_id=d0a5920e58894979837e43d1974873be, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:29.153863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56271e1f-c15c-4591-a3d9-39c16569b2bb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:29.162283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:29.170590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 817/1635] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:29 2022] PUT /identity/v3/projects/5602ec31052e4716845e3df6324a8d7a/users/d0a5920e58894979837e43d1974873be/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:29.172032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd6e92-0662-4470-b963-80d6396d6465 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:29.172386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd6e92-0662-4470-b963-80d6396d6465 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:29.172858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd6e92-0662-4470-b963-80d6396d6465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:29.202768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:29.209664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-efbd6e92-0662-4470-b963-80d6396d6465 None None] MFA Rules not processed for user `0ab7f9049c2a4511817c7e47107a9c87`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:29.211165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-efbd6e92-0662-4470-b963-80d6396d6465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:29.265503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21eae4e-1418-4a6a-b691-8c202918068d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:29.266687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21eae4e-1418-4a6a-b691-8c202918068d service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:29.266687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21eae4e-1418-4a6a-b691-8c202918068d service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:29.267127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21eae4e-1418-4a6a-b691-8c202918068d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:29.318851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 818/1636] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:29.318851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60ecd5-f378-43b6-951a-0eef2ee2701f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:29.318851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60ecd5-f378-43b6-951a-0eef2ee2701f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:29.318851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60ecd5-f378-43b6-951a-0eef2ee2701f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:29.374366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6a60ecd5-f378-43b6-951a-0eef2ee2701f None None] MFA Rules not processed for user `d0a5920e58894979837e43d1974873be`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:29.376683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6a60ecd5-f378-43b6-951a-0eef2ee2701f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:29.384124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21eae4e-1418-4a6a-b691-8c202918068d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:29.386206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21eae4e-1418-4a6a-b691-8c202918068d service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:29.397967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 819/1637] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:28 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:29.398967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21eae4e-1418-4a6a-b691-8c202918068d service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:29.473362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21eae4e-1418-4a6a-b691-8c202918068d service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:29.498894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 819/1638] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:29.498894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ca381-4ca5-4f0c-81be-3d1b425ad747 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:29.498894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ca381-4ca5-4f0c-81be-3d1b425ad747 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:29.499462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ca381-4ca5-4f0c-81be-3d1b425ad747 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:29.501265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 820/1639] 10.222.0.100 () {54 vars in 917 bytes} [Thu Sep 1 17:36:29 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:29.509361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca48856-70a7-4ed3-9036-03ab2ef2e226 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:29.509774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca48856-70a7-4ed3-9036-03ab2ef2e226 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:29.510441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca48856-70a7-4ed3-9036-03ab2ef2e226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:29.549636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cca48856-70a7-4ed3-9036-03ab2ef2e226 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:29.551941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e813de-cb07-46a9-a73e-d7124f1ed779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:29.552767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e813de-cb07-46a9-a73e-d7124f1ed779 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:29.553041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cca48856-70a7-4ed3-9036-03ab2ef2e226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:29.553290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e813de-cb07-46a9-a73e-d7124f1ed779 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:29.553660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e813de-cb07-46a9-a73e-d7124f1ed779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:29.565268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e813de-cb07-46a9-a73e-d7124f1ed779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:29.567077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e813de-cb07-46a9-a73e-d7124f1ed779 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:29.579778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 820/1640] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:29.671385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e813de-cb07-46a9-a73e-d7124f1ed779 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:29.688206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e813de-cb07-46a9-a73e-d7124f1ed779 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:29.700742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611983c-6129-452e-8497-f86cd2a44855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:29.701265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611983c-6129-452e-8497-f86cd2a44855 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:29.701552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611983c-6129-452e-8497-f86cd2a44855 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:29.703422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611983c-6129-452e-8497-f86cd2a44855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:29.728917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611983c-6129-452e-8497-f86cd2a44855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:29.734892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611983c-6129-452e-8497-f86cd2a44855 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:29.740212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 821/1641] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:36:29 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:36:29.785152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 821/1642] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:36:29.987835 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca48856-70a7-4ed3-9036-03ab2ef2e226 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:29.993599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:30.002794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca48856-70a7-4ed3-9036-03ab2ef2e226 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:30.013993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff778e8-d85f-47b9-8d15-359da69eba19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:30.016089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff778e8-d85f-47b9-8d15-359da69eba19 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:30.016399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff778e8-d85f-47b9-8d15-359da69eba19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:30.016682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff778e8-d85f-47b9-8d15-359da69eba19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:30.053251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff778e8-d85f-47b9-8d15-359da69eba19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:30.054759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff778e8-d85f-47b9-8d15-359da69eba19 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:30.081199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 822/1643] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:29 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:30.122374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611983c-6129-452e-8497-f86cd2a44855 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:30.133766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611983c-6129-452e-8497-f86cd2a44855 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:30.142910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc002b60-c68a-4d3f-ba20-5297f870dd12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:30.144625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc002b60-c68a-4d3f-ba20-5297f870dd12 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:30.145705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc002b60-c68a-4d3f-ba20-5297f870dd12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:30.146064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc002b60-c68a-4d3f-ba20-5297f870dd12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:30.221069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:30.293938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc002b60-c68a-4d3f-ba20-5297f870dd12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:30.295528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc002b60-c68a-4d3f-ba20-5297f870dd12 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:30.330004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 822/1644] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:30.690550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff778e8-d85f-47b9-8d15-359da69eba19 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:30.725859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff778e8-d85f-47b9-8d15-359da69eba19 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:30.734260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8083585-ef44-4d2c-a75e-72e505c02db9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:30.735114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8083585-ef44-4d2c-a75e-72e505c02db9 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:30.735604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8083585-ef44-4d2c-a75e-72e505c02db9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:30.735762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8083585-ef44-4d2c-a75e-72e505c02db9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:30.865666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8083585-ef44-4d2c-a75e-72e505c02db9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:30.871945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8083585-ef44-4d2c-a75e-72e505c02db9 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:30.896672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 823/1645] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:31.399737 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc002b60-c68a-4d3f-ba20-5297f870dd12 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:31.621795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc002b60-c68a-4d3f-ba20-5297f870dd12 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:31.626918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:31.627585 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:31.627870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:31.629005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] PATH_INFO: `/v3/users/781a9b0e524c46fc89908e1a1bc4c14c` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:31.629368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] RBAC: Authorizing `identity:delete_user(user_id=781a9b0e524c46fc89908e1a1bc4c14c)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:31.631447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:31.800616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '781a9b0e524c46fc89908e1a1bc4c14c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:31.817178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '781a9b0e524c46fc89908e1a1bc4c14c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:31.817845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '781a9b0e524c46fc89908e1a1bc4c14c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:31.848545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '781a9b0e524c46fc89908e1a1bc4c14c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:31.882166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '781a9b0e524c46fc89908e1a1bc4c14c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:32.062067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 823/1646] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:31 2022] DELETE /identity/v3/users/781a9b0e524c46fc89908e1a1bc4c14c => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:32.063418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc4738-4e1d-4448-8f52-b7c2953a6816 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:32.063418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc4738-4e1d-4448-8f52-b7c2953a6816 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:32.063418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc4738-4e1d-4448-8f52-b7c2953a6816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:32.063418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fcdc4738-4e1d-4448-8f52-b7c2953a6816 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:32.063418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fcdc4738-4e1d-4448-8f52-b7c2953a6816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:32.147942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 824/1647] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:32.181716 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:32.186977 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:32.407559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:32.423277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e294f72-9741-42a7-9bc0-cc464919caae admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:32.434660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b332b51-ceac-4376-9c4a-eb0456c562bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:32.437332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b332b51-ceac-4376-9c4a-eb0456c562bf service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:32.437899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b332b51-ceac-4376-9c4a-eb0456c562bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:32.438414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b332b51-ceac-4376-9c4a-eb0456c562bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:32.469736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b332b51-ceac-4376-9c4a-eb0456c562bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:32.476193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b332b51-ceac-4376-9c4a-eb0456c562bf service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:32.491120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 824/1648] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:33.599045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc4738-4e1d-4448-8f52-b7c2953a6816 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:33.600516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fcdc4738-4e1d-4448-8f52-b7c2953a6816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:33.623315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc4738-4e1d-4448-8f52-b7c2953a6816 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:33.653567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:33.655622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:33.656672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:33.657082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] PATH_INFO: `/v3/projects/49ce77df03d34465a8d809e9c82fb42f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:33.664377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49ce77df03d34465a8d809e9c82fb42f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:33.667873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:33.847078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49ce77df03d34465a8d809e9c82fb42f', 'request_id': 'req-464c05c2-474f-44c6-bbc9-16a256f37386'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:33.864805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49ce77df03d34465a8d809e9c82fb42f', 'request_id': 'req-464c05c2-474f-44c6-bbc9-16a256f37386'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:33.878692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49ce77df03d34465a8d809e9c82fb42f', 'request_id': 'req-464c05c2-474f-44c6-bbc9-16a256f37386'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:33.911262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] The token cache is being invalidate because project 49ce77df03d34465a8d809e9c82fb42f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:36:33.912064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:33.912786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-464c05c2-474f-44c6-bbc9-16a256f37386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:33.915525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 825/1649] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:36:33 2022] DELETE /identity/v3/projects/49ce77df03d34465a8d809e9c82fb42f => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:33.921030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb7ba6-6ee9-4a9a-a1e7-3967bfb81e5f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:33.922081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb7ba6-6ee9-4a9a-a1e7-3967bfb81e5f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:33.922539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb7ba6-6ee9-4a9a-a1e7-3967bfb81e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:34.009261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-94fb7ba6-6ee9-4a9a-a1e7-3967bfb81e5f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:34.009261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-94fb7ba6-6ee9-4a9a-a1e7-3967bfb81e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:34.152470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 825/1650] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:34.159941 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef7adc-5d15-4d1d-8323-47a4414d1372 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:34.160224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef7adc-5d15-4d1d-8323-47a4414d1372 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:34.160499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef7adc-5d15-4d1d-8323-47a4414d1372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:34.214001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0bef7adc-5d15-4d1d-8323-47a4414d1372 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:34.215341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0bef7adc-5d15-4d1d-8323-47a4414d1372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:34.243176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 826/1651] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:34.248296 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb7ba6-6ee9-4a9a-a1e7-3967bfb81e5f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:34.264169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb7ba6-6ee9-4a9a-a1e7-3967bfb81e5f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:34.282970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e3d9-55c6-47a2-a16e-8633f92a315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:34.283774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e3d9-55c6-47a2-a16e-8633f92a315f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:34.286916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e3d9-55c6-47a2-a16e-8633f92a315f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:34.287397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e3d9-55c6-47a2-a16e-8633f92a315f admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:34.290983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548e3d9-55c6-47a2-a16e-8633f92a315f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:34.291368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548e3d9-55c6-47a2-a16e-8633f92a315f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:34.294864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548e3d9-55c6-47a2-a16e-8633f92a315f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:34.343718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 826/1652] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:34.345109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef7adc-5d15-4d1d-8323-47a4414d1372 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:34.356592 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:34.374630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef7adc-5d15-4d1d-8323-47a4414d1372 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:34.404025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f331a9-ff45-4a7c-b74c-4636a31dc906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:34.407094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f331a9-ff45-4a7c-b74c-4636a31dc906 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:34.408644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f331a9-ff45-4a7c-b74c-4636a31dc906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:34.409019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f331a9-ff45-4a7c-b74c-4636a31dc906 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:34.410025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f331a9-ff45-4a7c-b74c-4636a31dc906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:34.413326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f331a9-ff45-4a7c-b74c-4636a31dc906 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:34.493412 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 827/1653] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:34 2022] POST /identity/v3/projects => generated 368 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:34.498031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e3d9-55c6-47a2-a16e-8633f92a315f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:34.516328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e3d9-55c6-47a2-a16e-8633f92a315f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:34.768465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c100-0dc8-4ef1-a593-69e141dfe086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:34.769496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c100-0dc8-4ef1-a593-69e141dfe086 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:34.769965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c100-0dc8-4ef1-a593-69e141dfe086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:34.770367 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c100-0dc8-4ef1-a593-69e141dfe086 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:34.771440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3c100-0dc8-4ef1-a593-69e141dfe086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:34.774953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3c100-0dc8-4ef1-a593-69e141dfe086 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:34.850175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 827/1654] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:34 2022] POST /identity/v3/users => generated 285 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:34.855036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f331a9-ff45-4a7c-b74c-4636a31dc906 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:34.877224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f331a9-ff45-4a7c-b74c-4636a31dc906 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:34.895865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a30536-ff50-41b2-a2d8-2458b87f96a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:34.896496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a30536-ff50-41b2-a2d8-2458b87f96a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:34.900346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a30536-ff50-41b2-a2d8-2458b87f96a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:34.903755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a30536-ff50-41b2-a2d8-2458b87f96a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:34.906254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a30536-ff50-41b2-a2d8-2458b87f96a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:34.914825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a30536-ff50-41b2-a2d8-2458b87f96a8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:34.967729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 828/1655] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:34 2022] GET /identity/v3/roles => generated 1427 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:34.969185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c100-0dc8-4ef1-a593-69e141dfe086 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:34.990377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c100-0dc8-4ef1-a593-69e141dfe086 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:35.024381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2691a-61cb-4875-9dc1-bf99c56e2c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:35.025033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2691a-61cb-4875-9dc1-bf99c56e2c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:35.025315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2691a-61cb-4875-9dc1-bf99c56e2c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:35.025604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2691a-61cb-4875-9dc1-bf99c56e2c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:35.026664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2691a-61cb-4875-9dc1-bf99c56e2c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:35.028790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2691a-61cb-4875-9dc1-bf99c56e2c2c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:35.043533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 828/1656] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:34 2022] GET /identity/v3/roles => generated 1427 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:35.047513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a30536-ff50-41b2-a2d8-2458b87f96a8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:35.080731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a30536-ff50-41b2-a2d8-2458b87f96a8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:35.109481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a31ea9-20c3-4cb2-8885-6d7751b110e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:35.110561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31ea9-20c3-4cb2-8885-6d7751b110e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:35.111467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31ea9-20c3-4cb2-8885-6d7751b110e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:35.114663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31ea9-20c3-4cb2-8885-6d7751b110e4 admin admin] PATH_INFO: `/v3/projects/9e7160aa6bee486db311a76a97d8e2fd/users/525ea29e58484f99b7d244d21cd53eac/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:35.152982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a31ea9-20c3-4cb2-8885-6d7751b110e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7160aa6bee486db311a76a97d8e2fd, user_id=525ea29e58484f99b7d244d21cd53eac, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:35.152982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a31ea9-20c3-4cb2-8885-6d7751b110e4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:35.175987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 829/1657] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:35 2022] PUT /identity/v3/projects/9e7160aa6bee486db311a76a97d8e2fd/users/525ea29e58484f99b7d244d21cd53eac/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:35.187760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f91d46-5758-4746-b45e-b4cfe03c7b1c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:35.188051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f91d46-5758-4746-b45e-b4cfe03c7b1c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:35.188329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f91d46-5758-4746-b45e-b4cfe03c7b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:35.245967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-01f91d46-5758-4746-b45e-b4cfe03c7b1c None None] MFA Rules not processed for user `525ea29e58484f99b7d244d21cd53eac`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:35.262079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-01f91d46-5758-4746-b45e-b4cfe03c7b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:35.407097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:35.440000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 829/1658] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:35.443887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a31ea9-20c3-4cb2-8885-6d7751b110e4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:35.589904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a31ea9-20c3-4cb2-8885-6d7751b110e4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:35.603743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f76991-d092-43fe-8cea-edd6dd578d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:35.605500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f76991-d092-43fe-8cea-edd6dd578d45 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:35.607719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f76991-d092-43fe-8cea-edd6dd578d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:35.608226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f76991-d092-43fe-8cea-edd6dd578d45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:35.611118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f76991-d092-43fe-8cea-edd6dd578d45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:35.616578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f76991-d092-43fe-8cea-edd6dd578d45 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:35.674577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 830/1659] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:35 2022] POST /identity/v3/projects => generated 368 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:35.683187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f91d46-5758-4746-b45e-b4cfe03c7b1c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:35.717643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f91d46-5758-4746-b45e-b4cfe03c7b1c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:35.881774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235109ba-4b24-46b4-9462-366fe3ae0215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:35.882966 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235109ba-4b24-46b4-9462-366fe3ae0215 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:35.883638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235109ba-4b24-46b4-9462-366fe3ae0215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:35.884514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235109ba-4b24-46b4-9462-366fe3ae0215 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:35.885188 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235109ba-4b24-46b4-9462-366fe3ae0215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:35.890755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235109ba-4b24-46b4-9462-366fe3ae0215 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:35.945893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 830/1660] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:35 2022] POST /identity/v3/users => generated 291 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:35.946004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f76991-d092-43fe-8cea-edd6dd578d45 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:35.966813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f76991-d092-43fe-8cea-edd6dd578d45 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:35.993130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8ab9a-147e-417e-b91c-ee7212925561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:35.993469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8ab9a-147e-417e-b91c-ee7212925561 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:35.995825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8ab9a-147e-417e-b91c-ee7212925561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:35.996487 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8ab9a-147e-417e-b91c-ee7212925561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:35.999376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8ab9a-147e-417e-b91c-ee7212925561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:36.003635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8ab9a-147e-417e-b91c-ee7212925561 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:36.066139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 831/1661] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:35 2022] GET /identity/v3/roles => generated 1427 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:36:36.069478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235109ba-4b24-46b4-9462-366fe3ae0215 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:36.074714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-235109ba-4b24-46b4-9462-366fe3ae0215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:36.075066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8ab9a-147e-417e-b91c-ee7212925561 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:36.094089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8ab9a-147e-417e-b91c-ee7212925561 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:36.099553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235109ba-4b24-46b4-9462-366fe3ae0215 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:36.100066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04003c40-244c-4e20-8560-084f9492e95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:36.100973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04003c40-244c-4e20-8560-084f9492e95b admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:36.101279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04003c40-244c-4e20-8560-084f9492e95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:36.101623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04003c40-244c-4e20-8560-084f9492e95b admin admin] PATH_INFO: `/v3/projects/7c348c6758f74a90b71931294b48cbb3/users/70b8e73b7dfa457295098327a6f0a6ce/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:36.105924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:36.106815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:36.107272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:36.108129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] PATH_INFO: `/v3/users/e7617a3afd7949daaa7763a784693ca2` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:36.109772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7617a3afd7949daaa7763a784693ca2)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:36.115475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:36.195277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04003c40-244c-4e20-8560-084f9492e95b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c348c6758f74a90b71931294b48cbb3, user_id=70b8e73b7dfa457295098327a6f0a6ce, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:36.200086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04003c40-244c-4e20-8560-084f9492e95b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:36.219058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 832/1662] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:36 2022] PUT /identity/v3/projects/7c348c6758f74a90b71931294b48cbb3/users/70b8e73b7dfa457295098327a6f0a6ce/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:36.224664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04003c40-244c-4e20-8560-084f9492e95b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:36.275972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7617a3afd7949daaa7763a784693ca2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:36.292242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7617a3afd7949daaa7763a784693ca2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:36.293236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7617a3afd7949daaa7763a784693ca2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:36.338044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7617a3afd7949daaa7763a784693ca2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:36.362505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:36.362906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-695ee1c3-018d-4601-a354-f46bdad43f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7617a3afd7949daaa7763a784693ca2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:36.371829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04003c40-244c-4e20-8560-084f9492e95b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:36.382874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 831/1663] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:36 2022] DELETE /identity/v3/users/e7617a3afd7949daaa7763a784693ca2 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:36.388946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcf70a-e4b3-44d7-ae14-baa11f3ddb93 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:36.389419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcf70a-e4b3-44d7-ae14-baa11f3ddb93 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:36.389924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcf70a-e4b3-44d7-ae14-baa11f3ddb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:36.393423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300f4ec-779a-4831-9f9d-d58e801a7804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:36.394083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300f4ec-779a-4831-9f9d-d58e801a7804 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:36.394365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300f4ec-779a-4831-9f9d-d58e801a7804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:36.394643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300f4ec-779a-4831-9f9d-d58e801a7804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:36.395294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300f4ec-779a-4831-9f9d-d58e801a7804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:36.408449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300f4ec-779a-4831-9f9d-d58e801a7804 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:36.456083 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-21fcf70a-e4b3-44d7-ae14-baa11f3ddb93 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:36.465902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-21fcf70a-e4b3-44d7-ae14-baa11f3ddb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:36.473048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 833/1664] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:36 2022] GET /identity/v3/roles => generated 1427 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:36.475070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300f4ec-779a-4831-9f9d-d58e801a7804 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:36.481188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d300f4ec-779a-4831-9f9d-d58e801a7804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:36.526172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300f4ec-779a-4831-9f9d-d58e801a7804 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:36.616045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:36.680898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 832/1665] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:36.702319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcf70a-e4b3-44d7-ae14-baa11f3ddb93 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:36.706485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-21fcf70a-e4b3-44d7-ae14-baa11f3ddb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:36.771384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b69ac7-d69a-4746-8bcb-1b13d32e0216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:36.772085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b69ac7-d69a-4746-8bcb-1b13d32e0216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:36.772369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b69ac7-d69a-4746-8bcb-1b13d32e0216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:36.772648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b69ac7-d69a-4746-8bcb-1b13d32e0216 admin admin] PATH_INFO: `/v3/projects/7c348c6758f74a90b71931294b48cbb3/users/70b8e73b7dfa457295098327a6f0a6ce/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:36.778706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b69ac7-d69a-4746-8bcb-1b13d32e0216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c348c6758f74a90b71931294b48cbb3, user_id=70b8e73b7dfa457295098327a6f0a6ce, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:36.787270 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b69ac7-d69a-4746-8bcb-1b13d32e0216 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:36.820776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 834/1666] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:36 2022] PUT /identity/v3/projects/7c348c6758f74a90b71931294b48cbb3/users/70b8e73b7dfa457295098327a6f0a6ce/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:36.832005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd141b0b-0302-4abc-b5c6-d5a143d2ef01 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:36.832275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd141b0b-0302-4abc-b5c6-d5a143d2ef01 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:36.832548 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd141b0b-0302-4abc-b5c6-d5a143d2ef01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:36.902786 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcf70a-e4b3-44d7-ae14-baa11f3ddb93 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:36.975004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dd141b0b-0302-4abc-b5c6-d5a143d2ef01 None None] MFA Rules not processed for user `70b8e73b7dfa457295098327a6f0a6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:36.976249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dd141b0b-0302-4abc-b5c6-d5a143d2ef01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:37.052531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5d8e2-140c-4827-b94b-508d5fe035b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:37.053368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5d8e2-140c-4827-b94b-508d5fe035b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:37.053769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5d8e2-140c-4827-b94b-508d5fe035b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:37.054144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5d8e2-140c-4827-b94b-508d5fe035b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:37.187132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5d8e2-140c-4827-b94b-508d5fe035b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:37.189240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5d8e2-140c-4827-b94b-508d5fe035b6 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:37.217790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 833/1667] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:37.235965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 835/1668] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:37.249494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5d8e2-140c-4827-b94b-508d5fe035b6 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:37.251301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-32c5d8e2-140c-4827-b94b-508d5fe035b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:37.430705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5d8e2-140c-4827-b94b-508d5fe035b6 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:37.442355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b6c53-3afd-4794-b13b-c3f074778cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:37.442994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b6c53-3afd-4794-b13b-c3f074778cea service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:37.443277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b6c53-3afd-4794-b13b-c3f074778cea service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:37.443551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b6c53-3afd-4794-b13b-c3f074778cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:37.465328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b6c53-3afd-4794-b13b-c3f074778cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:37.467523 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b6c53-3afd-4794-b13b-c3f074778cea service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:37.480179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:37.514512 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 834/1669] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:38.742109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02d82c-8a94-46db-a598-af6649732882 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:38.743171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02d82c-8a94-46db-a598-af6649732882 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:38.743677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02d82c-8a94-46db-a598-af6649732882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:38.800800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8b02d82c-8a94-46db-a598-af6649732882 None None] MFA Rules not processed for user `5647214c5b3f438eb3ef1ddc14153d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:38.802482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8b02d82c-8a94-46db-a598-af6649732882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:38.973710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 836/1670] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:38 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:38.984744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b4b0c-6792-43b1-b6e5-7812820fab4a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:38.985197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b4b0c-6792-43b1-b6e5-7812820fab4a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:38.985378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b4b0c-6792-43b1-b6e5-7812820fab4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:39.023067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fc8b4b0c-6792-43b1-b6e5-7812820fab4a None None] MFA Rules not processed for user `5647214c5b3f438eb3ef1ddc14153d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:39.024354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fc8b4b0c-6792-43b1-b6e5-7812820fab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:39.045615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 835/1671] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:38 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:39.065718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02d82c-8a94-46db-a598-af6649732882 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:39.067638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8b02d82c-8a94-46db-a598-af6649732882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:39.075134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:39.274285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02d82c-8a94-46db-a598-af6649732882 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:39.290344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053571ce-1e5d-458e-bbc9-fa30af1bcc93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:39.290994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053571ce-1e5d-458e-bbc9-fa30af1bcc93 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:39.291274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053571ce-1e5d-458e-bbc9-fa30af1bcc93 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:39.299556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053571ce-1e5d-458e-bbc9-fa30af1bcc93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:39.316802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053571ce-1e5d-458e-bbc9-fa30af1bcc93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:39.318574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053571ce-1e5d-458e-bbc9-fa30af1bcc93 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:39.345442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 837/1672] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:39 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:39.683855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b4b0c-6792-43b1-b6e5-7812820fab4a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:39.685621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fc8b4b0c-6792-43b1-b6e5-7812820fab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:39.718285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b4b0c-6792-43b1-b6e5-7812820fab4a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:39.740636 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:39.741290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:39.741558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:39.741864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] PATH_INFO: `/v3/projects/f67baaa5ea164be2b89d3289fc9225fa` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:39.746542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f67baaa5ea164be2b89d3289fc9225fa)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:39.747596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:36:39.747596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Sep 01 17:36:39.747596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: Sep 01 17:36:39.748841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:39.950288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f67baaa5ea164be2b89d3289fc9225fa', 'request_id': 'req-1a3e10e8-4ade-4d1c-b449-35548801cabc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:39.984623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f67baaa5ea164be2b89d3289fc9225fa', 'request_id': 'req-1a3e10e8-4ade-4d1c-b449-35548801cabc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:40.000917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f67baaa5ea164be2b89d3289fc9225fa', 'request_id': 'req-1a3e10e8-4ade-4d1c-b449-35548801cabc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:40.029182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] The token cache is being invalidate because project f67baaa5ea164be2b89d3289fc9225fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:36:40.029597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:40.037538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a3e10e8-4ade-4d1c-b449-35548801cabc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:40.046266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 836/1673] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:36:39 2022] DELETE /identity/v3/projects/f67baaa5ea164be2b89d3289fc9225fa => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:40.052000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322744e8-268e-486c-86ac-76bc06356067 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:40.052472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322744e8-268e-486c-86ac-76bc06356067 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:40.052556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322744e8-268e-486c-86ac-76bc06356067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:40.126240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-322744e8-268e-486c-86ac-76bc06356067 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:40.127503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-322744e8-268e-486c-86ac-76bc06356067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:40.423194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 838/1674] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:40.431822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323171dc-76b8-4036-95d5-ed0953d9b862 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:40.432037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323171dc-76b8-4036-95d5-ed0953d9b862 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:40.432614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323171dc-76b8-4036-95d5-ed0953d9b862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:40.476853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-323171dc-76b8-4036-95d5-ed0953d9b862 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:40.478541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-323171dc-76b8-4036-95d5-ed0953d9b862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:40.491005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 837/1675] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:40.495431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322744e8-268e-486c-86ac-76bc06356067 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:40.511146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322744e8-268e-486c-86ac-76bc06356067 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:40.535426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:40.536377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:40.538241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:40.538852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:40.543153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:40.543722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:40.548469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:40.607143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323171dc-76b8-4036-95d5-ed0953d9b862 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:40.608138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 839/1676] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:40.619581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:40.624140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:40.626610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323171dc-76b8-4036-95d5-ed0953d9b862 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:40.642049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8a243-bc0c-4621-afb1-6685d2730e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:40.643163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8a243-bc0c-4621-afb1-6685d2730e4f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:40.643619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8a243-bc0c-4621-afb1-6685d2730e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:40.644035 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8a243-bc0c-4621-afb1-6685d2730e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:40.645135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8a243-bc0c-4621-afb1-6685d2730e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:40.648131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8a243-bc0c-4621-afb1-6685d2730e4f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:40.676933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0794d0-a35f-4b20-845f-c25e685c39fb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:40.716919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:40.724074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:40.724369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:40.724653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] PATH_INFO: `/v3/users/f8ad7f0ec67542279d17b18e16da6bde` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:40.738317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ad7f0ec67542279d17b18e16da6bde)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:40.739418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:36:40.739418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Sep 01 17:36:40.739418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: Sep 01 17:36:40.740667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:40.783975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 838/1677] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:40 2022] POST /identity/v3/projects => generated 364 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:40.785724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c97b1d-a376-4388-879f-d3f37d819ac2 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:40.787534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c97b1d-a376-4388-879f-d3f37d819ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:40.787989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c97b1d-a376-4388-879f-d3f37d819ac2 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:40.801141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 839/1678] 10.222.0.100 () {54 vars in 917 bytes} [Thu Sep 1 17:36:40 2022] GET /identity => generated 271 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:40.802965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c97b1d-a376-4388-879f-d3f37d819ac2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:40.986135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ad7f0ec67542279d17b18e16da6bde'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.053751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ad7f0ec67542279d17b18e16da6bde'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.061208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c97b1d-a376-4388-879f-d3f37d819ac2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:41.067393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:41.080028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b7410-ec84-4453-970b-af2433389165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:41.081027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b7410-ec84-4453-970b-af2433389165 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:41.081313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b7410-ec84-4453-970b-af2433389165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:41.084151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b7410-ec84-4453-970b-af2433389165 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:41.084859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b7410-ec84-4453-970b-af2433389165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:41.089072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ad7f0ec67542279d17b18e16da6bde'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.089960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b7410-ec84-4453-970b-af2433389165 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:41.100826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ad7f0ec67542279d17b18e16da6bde'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.102049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b33dd013-c4c4-4858-92db-18de2381731e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ad7f0ec67542279d17b18e16da6bde'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.116159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 840/1679] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:40 2022] DELETE /identity/v3/users/f8ad7f0ec67542279d17b18e16da6bde => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:41.117682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adc08c-9085-4452-a264-ece96487193a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:41.117975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adc08c-9085-4452-a264-ece96487193a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:41.118251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adc08c-9085-4452-a264-ece96487193a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:41.154136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 840/1680] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:40 2022] POST /identity/v3/users => generated 283 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:41.156235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b7410-ec84-4453-970b-af2433389165 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:41.161548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e19b7410-ec84-4453-970b-af2433389165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:41.198389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b7410-ec84-4453-970b-af2433389165 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:41.214385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-49adc08c-9085-4452-a264-ece96487193a None None] MFA Rules not processed for user `bbe180b45f2e49e2abbf6e833ce0d44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:41.217539 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-49adc08c-9085-4452-a264-ece96487193a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:41.296125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:41.361766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 841/1681] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:36:41 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:41.398385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47dafd-8d80-4e10-b3f3-ff47562638ea None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:41.398855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47dafd-8d80-4e10-b3f3-ff47562638ea None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:41.399554 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47dafd-8d80-4e10-b3f3-ff47562638ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:41.400144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:41.400808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:41.401784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:41.404622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] PATH_INFO: `/v3/users/525ea29e58484f99b7d244d21cd53eac` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:41.410718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=525ea29e58484f99b7d244d21cd53eac)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:41.415137 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:41.436324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-be47dafd-8d80-4e10-b3f3-ff47562638ea None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:41.438046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-be47dafd-8d80-4e10-b3f3-ff47562638ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:41.456897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 842/1682] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:41.459356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47dafd-8d80-4e10-b3f3-ff47562638ea None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:41.467151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-be47dafd-8d80-4e10-b3f3-ff47562638ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:41.502507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47dafd-8d80-4e10-b3f3-ff47562638ea None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:41.520026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc26c5-5a93-44df-ad12-28387c46e9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:41.521537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc26c5-5a93-44df-ad12-28387c46e9dc admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:41.522352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc26c5-5a93-44df-ad12-28387c46e9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:41.523208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc26c5-5a93-44df-ad12-28387c46e9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:41.524680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc26c5-5a93-44df-ad12-28387c46e9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:41.527858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc26c5-5a93-44df-ad12-28387c46e9dc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:41.548109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 843/1683] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:41 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:41.549618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc26c5-5a93-44df-ad12-28387c46e9dc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:41.554881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9ffc26c5-5a93-44df-ad12-28387c46e9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:41.579496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc26c5-5a93-44df-ad12-28387c46e9dc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:41.589606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c9bd0-8274-4b49-8cf6-cc3682ceec91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:41.591163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c9bd0-8274-4b49-8cf6-cc3682ceec91 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:41.591813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c9bd0-8274-4b49-8cf6-cc3682ceec91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:41.592980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c9bd0-8274-4b49-8cf6-cc3682ceec91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:41.605342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525ea29e58484f99b7d244d21cd53eac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.612837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c9bd0-8274-4b49-8cf6-cc3682ceec91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:41.616121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c9bd0-8274-4b49-8cf6-cc3682ceec91 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:41.625508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525ea29e58484f99b7d244d21cd53eac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.629750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525ea29e58484f99b7d244d21cd53eac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.665725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 844/1684] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:41 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:41.670349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c9bd0-8274-4b49-8cf6-cc3682ceec91 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:41.684583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525ea29e58484f99b7d244d21cd53eac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.701625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c9bd0-8274-4b49-8cf6-cc3682ceec91 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:41.712785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525ea29e58484f99b7d244d21cd53eac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:41.723172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d764e-a1f6-4b35-b731-5ec76aa04079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:41.724184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d764e-a1f6-4b35-b731-5ec76aa04079 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:41.724714 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d764e-a1f6-4b35-b731-5ec76aa04079 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:41.725190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d764e-a1f6-4b35-b731-5ec76aa04079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:41.727924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d52d764e-a1f6-4b35-b731-5ec76aa04079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:41.728990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 841/1685] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:41 2022] DELETE /identity/v3/users/525ea29e58484f99b7d244d21cd53eac => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:41.733303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:41.746733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:41.755790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d764e-a1f6-4b35-b731-5ec76aa04079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:41.757624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d764e-a1f6-4b35-b731-5ec76aa04079 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:41.783089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 845/1686] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:41.784731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21c35f-48bc-4397-bc0b-9739b17e799c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:41.785264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21c35f-48bc-4397-bc0b-9739b17e799c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:41.785779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21c35f-48bc-4397-bc0b-9739b17e799c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:41.825220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2d21c35f-48bc-4397-bc0b-9739b17e799c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:41.827989 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d21c35f-48bc-4397-bc0b-9739b17e799c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:41.926250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5ce40-d797-46b8-af5d-5992be480896 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:41.939742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0d063-50e5-4e7e-83ae-2003053ca65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:41.940375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0d063-50e5-4e7e-83ae-2003053ca65f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:41.940657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0d063-50e5-4e7e-83ae-2003053ca65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:41.940931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0d063-50e5-4e7e-83ae-2003053ca65f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:41.941665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0d063-50e5-4e7e-83ae-2003053ca65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:41.943231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0d063-50e5-4e7e-83ae-2003053ca65f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:41.944718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 846/1687] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:41.962586 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 842/1688] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:41 2022] GET /identity/v3/roles => generated 1427 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:41.977600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21c35f-48bc-4397-bc0b-9739b17e799c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:42.004568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0d063-50e5-4e7e-83ae-2003053ca65f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:42.014219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cca0d063-50e5-4e7e-83ae-2003053ca65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:42.017147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21c35f-48bc-4397-bc0b-9739b17e799c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:42.039574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12352b56-cd5a-48e7-bcf7-0e03a0911532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:42.047583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12352b56-cd5a-48e7-bcf7-0e03a0911532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:42.049713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12352b56-cd5a-48e7-bcf7-0e03a0911532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:42.052436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12352b56-cd5a-48e7-bcf7-0e03a0911532 admin admin] PATH_INFO: `/v3/projects/99517d122b434216838fcafc141e1ba2/users/a178bbfe5892460f973410f0a22b2b8d/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:42.142404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12352b56-cd5a-48e7-bcf7-0e03a0911532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99517d122b434216838fcafc141e1ba2, user_id=a178bbfe5892460f973410f0a22b2b8d, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:42.156340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12352b56-cd5a-48e7-bcf7-0e03a0911532 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:42.210066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 847/1689] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:41 2022] PUT /identity/v3/projects/99517d122b434216838fcafc141e1ba2/users/a178bbfe5892460f973410f0a22b2b8d/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:42.224576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84656c6-8c10-4635-9e38-ba17c2c5224c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:42.225054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84656c6-8c10-4635-9e38-ba17c2c5224c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:42.225683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84656c6-8c10-4635-9e38-ba17c2c5224c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:42.295631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0d063-50e5-4e7e-83ae-2003053ca65f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:42.301469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:42.346275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a84656c6-8c10-4635-9e38-ba17c2c5224c None None] MFA Rules not processed for user `a178bbfe5892460f973410f0a22b2b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:42.347543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a84656c6-8c10-4635-9e38-ba17c2c5224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:42.354929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:42.407031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfff3ec-c606-4e83-80b8-ed4c619b2f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:42.407667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfff3ec-c606-4e83-80b8-ed4c619b2f82 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:42.409771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfff3ec-c606-4e83-80b8-ed4c619b2f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:42.410215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfff3ec-c606-4e83-80b8-ed4c619b2f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:42.567358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 848/1690] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:42.572525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84656c6-8c10-4635-9e38-ba17c2c5224c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:42.624971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfff3ec-c606-4e83-80b8-ed4c619b2f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:42.645287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfff3ec-c606-4e83-80b8-ed4c619b2f82 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:42.716531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 843/1691] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:42.810380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84656c6-8c10-4635-9e38-ba17c2c5224c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:42.835373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b56639-16a9-40d0-8bd0-a9020160c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:42.836089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b56639-16a9-40d0-8bd0-a9020160c336 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:42.836415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b56639-16a9-40d0-8bd0-a9020160c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:42.836781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b56639-16a9-40d0-8bd0-a9020160c336 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:42.837478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b56639-16a9-40d0-8bd0-a9020160c336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:42.845856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b56639-16a9-40d0-8bd0-a9020160c336 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:42.956687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 849/1692] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:42 2022] POST /identity/v3/projects => generated 364 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:42.964373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfff3ec-c606-4e83-80b8-ed4c619b2f82 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:42.979044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b56639-16a9-40d0-8bd0-a9020160c336 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:42.984691 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfff3ec-c606-4e83-80b8-ed4c619b2f82 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:42.987816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-54b56639-16a9-40d0-8bd0-a9020160c336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:43.175394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125485d-231b-4dda-873d-3331c98d2d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:43.176404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125485d-231b-4dda-873d-3331c98d2d94 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:43.178177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125485d-231b-4dda-873d-3331c98d2d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:43.178626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125485d-231b-4dda-873d-3331c98d2d94 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:43.179441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125485d-231b-4dda-873d-3331c98d2d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:43.181428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125485d-231b-4dda-873d-3331c98d2d94 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:43.228827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 844/1693] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:42 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:43.228827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125485d-231b-4dda-873d-3331c98d2d94 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:43.248693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125485d-231b-4dda-873d-3331c98d2d94 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:43.253234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b56639-16a9-40d0-8bd0-a9020160c336 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:43.269963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a0869-79ec-4e36-ab92-d0ce904a48f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:43.270592 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0869-79ec-4e36-ab92-d0ce904a48f2 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:43.270867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0869-79ec-4e36-ab92-d0ce904a48f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:43.271479 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0869-79ec-4e36-ab92-d0ce904a48f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:43.273746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23588dde-c928-4e14-9cd8-48b172f0de48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:43.275514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23588dde-c928-4e14-9cd8-48b172f0de48 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:43.276953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23588dde-c928-4e14-9cd8-48b172f0de48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:43.277408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23588dde-c928-4e14-9cd8-48b172f0de48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:43.278332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23588dde-c928-4e14-9cd8-48b172f0de48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:43.279894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dd6a0869-79ec-4e36-ab92-d0ce904a48f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:43.280900 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23588dde-c928-4e14-9cd8-48b172f0de48 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:43.300683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 845/1694] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:43 2022] GET /identity/v3/roles => generated 1427 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:43.303342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23588dde-c928-4e14-9cd8-48b172f0de48 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:43.303552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a0869-79ec-4e36-ab92-d0ce904a48f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:43.305171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a0869-79ec-4e36-ab92-d0ce904a48f2 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:43.316837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23588dde-c928-4e14-9cd8-48b172f0de48 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:43.326688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dd7e6-7582-409d-b365-18b005bf3bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:43.329295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dd7e6-7582-409d-b365-18b005bf3bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:43.329607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dd7e6-7582-409d-b365-18b005bf3bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:43.330214 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dd7e6-7582-409d-b365-18b005bf3bd8 admin admin] PATH_INFO: `/v3/projects/6f27612fe55545439488b921408683db/users/c47467954c4842a6acb4261f2d3544c9/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:43.337642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 850/1695] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:42 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:43.409615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dd7e6-7582-409d-b365-18b005bf3bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f27612fe55545439488b921408683db, user_id=c47467954c4842a6acb4261f2d3544c9, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:43.409615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dd7e6-7582-409d-b365-18b005bf3bd8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:43.424099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc3846-09a3-48ae-abc9-31eeabc8d625 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:43.424392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc3846-09a3-48ae-abc9-31eeabc8d625 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:43.424669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc3846-09a3-48ae-abc9-31eeabc8d625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:43.464455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 846/1696] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:43 2022] PUT /identity/v3/projects/6f27612fe55545439488b921408683db/users/c47467954c4842a6acb4261f2d3544c9/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:43.464771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-89fc3846-09a3-48ae-abc9-31eeabc8d625 None None] MFA Rules not processed for user `5647214c5b3f438eb3ef1ddc14153d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:43.466158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dd7e6-7582-409d-b365-18b005bf3bd8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:43.469064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-89fc3846-09a3-48ae-abc9-31eeabc8d625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:43.495691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:43.526955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:43.611883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 851/1697] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:43 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:43.786623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dd7e6-7582-409d-b365-18b005bf3bd8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:43.804635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaae9ef-850c-4644-b793-71d4e7919cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:43.805559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae9ef-850c-4644-b793-71d4e7919cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:43.805916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae9ef-850c-4644-b793-71d4e7919cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:43.806283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae9ef-850c-4644-b793-71d4e7919cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:43.807906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaae9ef-850c-4644-b793-71d4e7919cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:43.811869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaae9ef-850c-4644-b793-71d4e7919cb4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:43.842935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 847/1698] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:43 2022] GET /identity/v3/roles => generated 1427 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:43.843075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc3846-09a3-48ae-abc9-31eeabc8d625 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:43.861252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc3846-09a3-48ae-abc9-31eeabc8d625 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:43.885069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dd14e-63ec-4ed1-ba5e-7c61b7accf8e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:43.886255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dd14e-63ec-4ed1-ba5e-7c61b7accf8e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:43.887009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dd14e-63ec-4ed1-ba5e-7c61b7accf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:43.887226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed46f73-733f-4f2c-bc6d-791f738c9538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:43.887912 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed46f73-733f-4f2c-bc6d-791f738c9538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:43.888251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed46f73-733f-4f2c-bc6d-791f738c9538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:43.888557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed46f73-733f-4f2c-bc6d-791f738c9538 admin admin] PATH_INFO: `/v3/projects/6f27612fe55545439488b921408683db/users/c47467954c4842a6acb4261f2d3544c9/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:43.899414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed46f73-733f-4f2c-bc6d-791f738c9538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f27612fe55545439488b921408683db, user_id=c47467954c4842a6acb4261f2d3544c9, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:43.902701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed46f73-733f-4f2c-bc6d-791f738c9538 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:43.934775 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 852/1699] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:43 2022] PUT /identity/v3/projects/6f27612fe55545439488b921408683db/users/c47467954c4842a6acb4261f2d3544c9/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:36:43.936913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed46f73-733f-4f2c-bc6d-791f738c9538 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:43.938633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6ed46f73-733f-4f2c-bc6d-791f738c9538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:43.967591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-966dd14e-63ec-4ed1-ba5e-7c61b7accf8e None None] MFA Rules not processed for user `bcea72e4fc77431a96e7acf9d68eb083`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:43.980433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-966dd14e-63ec-4ed1-ba5e-7c61b7accf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:44.099085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed46f73-733f-4f2c-bc6d-791f738c9538 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:44.113058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:44.113738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:44.114919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:44.115346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] PATH_INFO: `/v3/projects/19a5df264e224d8b901f524b41268ffc` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:44.118447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a5df264e224d8b901f524b41268ffc)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:44.121681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:44.153255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 848/1700] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:36:44.156206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee505c-ac4f-436a-8c79-a16075759dce None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:44.158910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee505c-ac4f-436a-8c79-a16075759dce None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:44.159356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee505c-ac4f-436a-8c79-a16075759dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:44.227948 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-eaee505c-ac4f-436a-8c79-a16075759dce None None] MFA Rules not processed for user `c47467954c4842a6acb4261f2d3544c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:44.230300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-eaee505c-ac4f-436a-8c79-a16075759dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:44.274225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a5df264e224d8b901f524b41268ffc', 'request_id': 'req-b60239f4-9c68-48c8-bdd7-92d636d50896'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:44.291140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a5df264e224d8b901f524b41268ffc', 'request_id': 'req-b60239f4-9c68-48c8-bdd7-92d636d50896'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:44.317055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a5df264e224d8b901f524b41268ffc', 'request_id': 'req-b60239f4-9c68-48c8-bdd7-92d636d50896'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:44.358442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] The token cache is being invalidate because project 19a5df264e224d8b901f524b41268ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:36:44.360049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:44.366334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b60239f4-9c68-48c8-bdd7-92d636d50896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:44.376380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 853/1701] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:36:43 2022] DELETE /identity/v3/projects/19a5df264e224d8b901f524b41268ffc => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:44.376380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8241e6-2adf-4d7c-91a7-e8c040af122f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:44.376380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8241e6-2adf-4d7c-91a7-e8c040af122f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:44.376380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8241e6-2adf-4d7c-91a7-e8c040af122f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:44.426630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bd8241e6-2adf-4d7c-91a7-e8c040af122f None None] MFA Rules not processed for user `bcea72e4fc77431a96e7acf9d68eb083`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:44.427893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bd8241e6-2adf-4d7c-91a7-e8c040af122f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:44.527923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:44.562366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 849/1702] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:44.571138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 854/1703] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:44.572362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8241e6-2adf-4d7c-91a7-e8c040af122f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:44.572893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64b43-7776-414e-995a-eb8754b2455d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:44.576326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64b43-7776-414e-995a-eb8754b2455d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:44.576899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bd8241e6-2adf-4d7c-91a7-e8c040af122f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:44.577261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64b43-7776-414e-995a-eb8754b2455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:44.658697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-43e64b43-7776-414e-995a-eb8754b2455d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:44.664959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-43e64b43-7776-414e-995a-eb8754b2455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:44.718008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:44.746904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8241e6-2adf-4d7c-91a7-e8c040af122f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:44.772883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:44.773709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:44.774015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:44.774292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] PATH_INFO: `/v3/projects/9e7160aa6bee486db311a76a97d8e2fd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:44.775692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7160aa6bee486db311a76a97d8e2fd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:44.778683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:44.877739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 850/1704] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:44.877739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e64b43-7776-414e-995a-eb8754b2455d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:44.878853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-43e64b43-7776-414e-995a-eb8754b2455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:44.954238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7160aa6bee486db311a76a97d8e2fd', 'request_id': 'req-f56cf772-65ae-40d2-87e2-af9a99f999b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:44.970360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7160aa6bee486db311a76a97d8e2fd', 'request_id': 'req-f56cf772-65ae-40d2-87e2-af9a99f999b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:44.977913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7160aa6bee486db311a76a97d8e2fd', 'request_id': 'req-f56cf772-65ae-40d2-87e2-af9a99f999b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:45.015764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] The token cache is being invalidate because project 9e7160aa6bee486db311a76a97d8e2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:36:45.015955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:45.017009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:45.019835 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 855/1705] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:36:44 2022] DELETE /identity/v3/projects/9e7160aa6bee486db311a76a97d8e2fd => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:45.022790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:45.028555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:45.100914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e64b43-7776-414e-995a-eb8754b2455d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:45.254853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cf772-65ae-40d2-87e2-af9a99f999b1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:45.264516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:45.265428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:45.265894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:45.266311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:45.268370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:45.289734 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506392d-ca73-4b89-b9fd-5cf7d638ca16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:45.290458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506392d-ca73-4b89-b9fd-5cf7d638ca16 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:45.290805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506392d-ca73-4b89-b9fd-5cf7d638ca16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:45.291147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506392d-ca73-4b89-b9fd-5cf7d638ca16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:45.303240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5506392d-ca73-4b89-b9fd-5cf7d638ca16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:45.305490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:45.451738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:45.456501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:45.480455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506392d-ca73-4b89-b9fd-5cf7d638ca16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:45.482136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506392d-ca73-4b89-b9fd-5cf7d638ca16 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:45.499117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 851/1706] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:45.514370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ecdb0-1e20-4a2d-9613-211a419baf82 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:45.514758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ecdb0-1e20-4a2d-9613-211a419baf82 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:45.514921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ecdb0-1e20-4a2d-9613-211a419baf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:45.532062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 856/1707] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:45.547359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:45.554434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:45.633804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-973ecdb0-1e20-4a2d-9613-211a419baf82 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:45.637446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-973ecdb0-1e20-4a2d-9613-211a419baf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:45.753875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:45.843773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 852/1708] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:45.845240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb024c6-55cf-4239-a1a6-4695e333e2d0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:45.846249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb024c6-55cf-4239-a1a6-4695e333e2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:45.846532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb024c6-55cf-4239-a1a6-4695e333e2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:45.969068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-adb024c6-55cf-4239-a1a6-4695e333e2d0 None None] MFA Rules not processed for user `bcc538bc852f404699d91d17023ae620`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:45.971498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-adb024c6-55cf-4239-a1a6-4695e333e2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:45.984239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceb9fc-0914-4249-ac5b-e9e212b87ef8 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:46.018577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:46.019465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:46.020057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:46.020477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] PATH_INFO: `/v3/users/70b8e73b7dfa457295098327a6f0a6ce` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:46.022300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b8e73b7dfa457295098327a6f0a6ce)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:46.033052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:46.196249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b8e73b7dfa457295098327a6f0a6ce'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:46.228562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b8e73b7dfa457295098327a6f0a6ce'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:46.250443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b8e73b7dfa457295098327a6f0a6ce'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:46.272423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b8e73b7dfa457295098327a6f0a6ce'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:46.273275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b8e73b7dfa457295098327a6f0a6ce'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:46.304179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 857/1709] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:45 2022] DELETE /identity/v3/users/70b8e73b7dfa457295098327a6f0a6ce => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:46.304179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:46.320111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:46.333910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 853/1710] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:46.353491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb024c6-55cf-4239-a1a6-4695e333e2d0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:46.358363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-adb024c6-55cf-4239-a1a6-4695e333e2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:46.433224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb024c6-55cf-4239-a1a6-4695e333e2d0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:46.486539 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:46.553947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378dcdb-33ea-4bf0-8d98-81522575eaac admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:46.562372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819d82b-3b9e-4eff-8210-faa4d421af28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:46.565390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819d82b-3b9e-4eff-8210-faa4d421af28 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:46.565805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819d82b-3b9e-4eff-8210-faa4d421af28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:46.566129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819d82b-3b9e-4eff-8210-faa4d421af28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:46.566814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819d82b-3b9e-4eff-8210-faa4d421af28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:46.567174 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819d82b-3b9e-4eff-8210-faa4d421af28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:46.572250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819d82b-3b9e-4eff-8210-faa4d421af28 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:46.619974 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 858/1711] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:46.627234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819d82b-3b9e-4eff-8210-faa4d421af28 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:46.678366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819d82b-3b9e-4eff-8210-faa4d421af28 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:46.680667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6c373-808c-4890-83f0-c33e97f0e148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:46.681358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6c373-808c-4890-83f0-c33e97f0e148 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:46.681853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6c373-808c-4890-83f0-c33e97f0e148 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:46.682768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6c373-808c-4890-83f0-c33e97f0e148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:46.688751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-03b6c373-808c-4890-83f0-c33e97f0e148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:46.699018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0bd4c-5a5c-4a03-b5e5-9e3fc342cc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:46.700654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0bd4c-5a5c-4a03-b5e5-9e3fc342cc79 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:46.701210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0bd4c-5a5c-4a03-b5e5-9e3fc342cc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:46.703706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0bd4c-5a5c-4a03-b5e5-9e3fc342cc79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:46.704577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0bd4c-5a5c-4a03-b5e5-9e3fc342cc79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:46.706646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0bd4c-5a5c-4a03-b5e5-9e3fc342cc79 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:46.796536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 859/1712] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:46 2022] POST /identity/v3/projects => generated 406 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:46.801949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa1bb-568a-4262-a44d-ab1c5212ba10 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:46.803026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa1bb-568a-4262-a44d-ab1c5212ba10 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:46.803166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa1bb-568a-4262-a44d-ab1c5212ba10 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:46.816935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 860/1713] 10.222.0.100 () {54 vars in 917 bytes} [Thu Sep 1 17:36:46 2022] GET /identity => generated 271 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:46.817993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa1bb-568a-4262-a44d-ab1c5212ba10 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:46.887792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:46.908925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6c373-808c-4890-83f0-c33e97f0e148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:46.913620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6c373-808c-4890-83f0-c33e97f0e148 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:46.952771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 854/1714] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:46.959460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a1a1b-7f23-45eb-9a88-d0ac71ad7635 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:46.959850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a1a1b-7f23-45eb-9a88-d0ac71ad7635 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:46.960607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a1a1b-7f23-45eb-9a88-d0ac71ad7635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:47.007779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-248a1a1b-7f23-45eb-9a88-d0ac71ad7635 None None] MFA Rules not processed for user `bbe180b45f2e49e2abbf6e833ce0d44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:47.014028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-248a1a1b-7f23-45eb-9a88-d0ac71ad7635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:47.064147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa1bb-568a-4262-a44d-ab1c5212ba10 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:47.099555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d5084-77e4-41fb-a59b-0be6f60ca21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:47.104475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d5084-77e4-41fb-a59b-0be6f60ca21f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:47.104781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d5084-77e4-41fb-a59b-0be6f60ca21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:47.109727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d5084-77e4-41fb-a59b-0be6f60ca21f admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:47.111850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d5084-77e4-41fb-a59b-0be6f60ca21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:47.121315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d5084-77e4-41fb-a59b-0be6f60ca21f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:47.198455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 855/1715] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:36:46 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:47.198455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a1a1b-7f23-45eb-9a88-d0ac71ad7635 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:47.198455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-248a1a1b-7f23-45eb-9a88-d0ac71ad7635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:47.199104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 861/1716] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:46 2022] POST /identity/v3/users => generated 304 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:47.199104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d5084-77e4-41fb-a59b-0be6f60ca21f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:47.202675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c05d5084-77e4-41fb-a59b-0be6f60ca21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:47.275660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d5084-77e4-41fb-a59b-0be6f60ca21f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:47.278277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a1a1b-7f23-45eb-9a88-d0ac71ad7635 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:47.300473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:47.300742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:47.300943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:47.301223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] PATH_INFO: `/v3/users/a178bbfe5892460f973410f0a22b2b8d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:47.304155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:47.306012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:47.306608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:47.306608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] PATH_INFO: `/v3/projects/7c348c6758f74a90b71931294b48cbb3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:47.309642 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a178bbfe5892460f973410f0a22b2b8d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:47.316247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c348c6758f74a90b71931294b48cbb3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:47.321073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:47.321761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:47.537505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c348c6758f74a90b71931294b48cbb3', 'request_id': 'req-85aff724-e762-4d61-a2ae-ef85d9054a8e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.556483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c348c6758f74a90b71931294b48cbb3', 'request_id': 'req-85aff724-e762-4d61-a2ae-ef85d9054a8e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.567621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c348c6758f74a90b71931294b48cbb3', 'request_id': 'req-85aff724-e762-4d61-a2ae-ef85d9054a8e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.575402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a178bbfe5892460f973410f0a22b2b8d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.607211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a178bbfe5892460f973410f0a22b2b8d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.607960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a178bbfe5892460f973410f0a22b2b8d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.624341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] The token cache is being invalidate because project 7c348c6758f74a90b71931294b48cbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:36:47.624668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.629324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.639414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 862/1717] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:36:47 2022] DELETE /identity/v3/projects/7c348c6758f74a90b71931294b48cbb3 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:47.640440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:47.647988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:47.668695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a178bbfe5892460f973410f0a22b2b8d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.698286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a178bbfe5892460f973410f0a22b2b8d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:47.721291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 856/1718] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:47 2022] DELETE /identity/v3/users/a178bbfe5892460f973410f0a22b2b8d => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:47.722760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:47.730376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:47.734938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:47.884185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aff724-e762-4d61-a2ae-ef85d9054a8e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:48.005981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d953b-d5fd-42c4-9374-af2305ef3af4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:48.022263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8c544-63fc-448b-af7e-200bb06e03ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:48.022902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8c544-63fc-448b-af7e-200bb06e03ee service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.023263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8c544-63fc-448b-af7e-200bb06e03ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.023544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8c544-63fc-448b-af7e-200bb06e03ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.035431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-74b8c544-63fc-448b-af7e-200bb06e03ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:48.068256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8c544-63fc-448b-af7e-200bb06e03ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:48.069680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8c544-63fc-448b-af7e-200bb06e03ee service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:48.092365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326772c3-866e-4dba-b139-81fe0f56afc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:48.095587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326772c3-866e-4dba-b139-81fe0f56afc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.096635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326772c3-866e-4dba-b139-81fe0f56afc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.097973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326772c3-866e-4dba-b139-81fe0f56afc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.100288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326772c3-866e-4dba-b139-81fe0f56afc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:48.105243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326772c3-866e-4dba-b139-81fe0f56afc9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:48.109330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 857/1719] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:47 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:48.113267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672627ad-1370-4417-ae3b-9af6a7079c9e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.117592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672627ad-1370-4417-ae3b-9af6a7079c9e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.120089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672627ad-1370-4417-ae3b-9af6a7079c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.148190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 863/1720] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:47 2022] GET /identity/v3/roles => generated 1427 bytes in 508 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:48.150218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21881a98-79d4-4b4b-8e8e-d54e87207653 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.150890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21881a98-79d4-4b4b-8e8e-d54e87207653 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.152630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21881a98-79d4-4b4b-8e8e-d54e87207653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.161728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-672627ad-1370-4417-ae3b-9af6a7079c9e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:48.163189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-672627ad-1370-4417-ae3b-9af6a7079c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:48.207246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 858/1721] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:48.215448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255bca4-f8a6-43f2-a0c1-168aa7a308f8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.215731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255bca4-f8a6-43f2-a0c1-168aa7a308f8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.224729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255bca4-f8a6-43f2-a0c1-168aa7a308f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.264806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-21881a98-79d4-4b4b-8e8e-d54e87207653 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:48.267745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-21881a98-79d4-4b4b-8e8e-d54e87207653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:48.301733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 864/1722] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:48.303277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21881a98-79d4-4b4b-8e8e-d54e87207653 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:48.310892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-21881a98-79d4-4b4b-8e8e-d54e87207653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:48.342096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:48.362257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21881a98-79d4-4b4b-8e8e-d54e87207653 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:48.390033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0449a5-3e15-4ae1-8bbe-e46ea044fdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:48.390994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0449a5-3e15-4ae1-8bbe-e46ea044fdca admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.391786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0449a5-3e15-4ae1-8bbe-e46ea044fdca admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.394440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0449a5-3e15-4ae1-8bbe-e46ea044fdca admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.398529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0449a5-3e15-4ae1-8bbe-e46ea044fdca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:48.399913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9255bca4-f8a6-43f2-a0c1-168aa7a308f8 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:48.401497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9255bca4-f8a6-43f2-a0c1-168aa7a308f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:48.401696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0449a5-3e15-4ae1-8bbe-e46ea044fdca admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:48.445954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 865/1723] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:48 2022] GET /identity/v3/roles => generated 1427 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:48.447934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7951-6743-40bd-a35f-ba55b7b47cae None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.448151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7951-6743-40bd-a35f-ba55b7b47cae None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.448424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7951-6743-40bd-a35f-ba55b7b47cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.524229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f41b7951-6743-40bd-a35f-ba55b7b47cae None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:48.535890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f41b7951-6743-40bd-a35f-ba55b7b47cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:48.567809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 866/1724] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:48.567809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b7951-6743-40bd-a35f-ba55b7b47cae None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:48.597755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b7951-6743-40bd-a35f-ba55b7b47cae None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:48.624866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a764-8884-4c7a-87a2-b8c6314a054e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:48.625510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a764-8884-4c7a-87a2-b8c6314a054e service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.625807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a764-8884-4c7a-87a2-b8c6314a054e service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.626078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a764-8884-4c7a-87a2-b8c6314a054e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.631938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 859/1725] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:48.632923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255bca4-f8a6-43f2-a0c1-168aa7a308f8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:48.659854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a764-8884-4c7a-87a2-b8c6314a054e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:48.671740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a764-8884-4c7a-87a2-b8c6314a054e service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:48.699988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255bca4-f8a6-43f2-a0c1-168aa7a308f8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:48.705593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 867/1726] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:48.706487 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a764-8884-4c7a-87a2-b8c6314a054e service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:48.719549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833cde0-f1fd-48d2-943f-3da1eecb1eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:48.721187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833cde0-f1fd-48d2-943f-3da1eecb1eec admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.721707 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833cde0-f1fd-48d2-943f-3da1eecb1eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.722161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833cde0-f1fd-48d2-943f-3da1eecb1eec admin admin] PATH_INFO: `/v3/projects/6ad6aff83c6b40938f933abc30b6c286/users/ded1577ffe054ede888ced02f099beeb/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.723732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a764-8884-4c7a-87a2-b8c6314a054e service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:48.740636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fe7dd-1a8d-405b-b4e9-eee515c4bad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:48.741443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe7dd-1a8d-405b-b4e9-eee515c4bad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.741758 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe7dd-1a8d-405b-b4e9-eee515c4bad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.742026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe7dd-1a8d-405b-b4e9-eee515c4bad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.746539 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fe7dd-1a8d-405b-b4e9-eee515c4bad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:48.746856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fe7dd-1a8d-405b-b4e9-eee515c4bad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:48.763126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833cde0-f1fd-48d2-943f-3da1eecb1eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ad6aff83c6b40938f933abc30b6c286, user_id=ded1577ffe054ede888ced02f099beeb, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:48.763687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fe7dd-1a8d-405b-b4e9-eee515c4bad4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:48.766571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833cde0-f1fd-48d2-943f-3da1eecb1eec admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:48.781362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 868/1727] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:48.784844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fe7dd-1a8d-405b-b4e9-eee515c4bad4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:48.794859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 860/1728] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:48 2022] PUT /identity/v3/projects/6ad6aff83c6b40938f933abc30b6c286/users/ded1577ffe054ede888ced02f099beeb/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:48.800199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fe7dd-1a8d-405b-b4e9-eee515c4bad4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:48.810443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005e6c0-ef95-4b93-b4b7-7565c6c03060 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.810961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005e6c0-ef95-4b93-b4b7-7565c6c03060 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.811400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005e6c0-ef95-4b93-b4b7-7565c6c03060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.875937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5005e6c0-ef95-4b93-b4b7-7565c6c03060 None None] MFA Rules not processed for user `ded1577ffe054ede888ced02f099beeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:48.877711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5005e6c0-ef95-4b93-b4b7-7565c6c03060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:48.988918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fdf51-777b-4898-98d4-011bd7889cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:48.989549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fdf51-777b-4898-98d4-011bd7889cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:48.989852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fdf51-777b-4898-98d4-011bd7889cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:48.990121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fdf51-777b-4898-98d4-011bd7889cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:48.990761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fdf51-777b-4898-98d4-011bd7889cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:48.992849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fdf51-777b-4898-98d4-011bd7889cd0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:49.061256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:49.090497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 869/1729] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:48 2022] POST /identity/v3/projects => generated 380 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:36:49.091769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fdf51-777b-4898-98d4-011bd7889cd0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:49.096179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bf5fdf51-777b-4898-98d4-011bd7889cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:49.161562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 861/1730] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3292 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:36:49.164144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005e6c0-ef95-4b93-b4b7-7565c6c03060 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:49.214006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005e6c0-ef95-4b93-b4b7-7565c6c03060 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:49.336699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:49.355656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daad897-1e2b-4108-a7ce-e634e147dae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:49.356301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daad897-1e2b-4108-a7ce-e634e147dae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:49.356758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daad897-1e2b-4108-a7ce-e634e147dae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:49.357032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daad897-1e2b-4108-a7ce-e634e147dae3 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:49.358891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daad897-1e2b-4108-a7ce-e634e147dae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:49.360887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daad897-1e2b-4108-a7ce-e634e147dae3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:49.433101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 862/1731] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:49 2022] POST /identity/v3/users => generated 291 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:49.434600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daad897-1e2b-4108-a7ce-e634e147dae3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:49.466728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daad897-1e2b-4108-a7ce-e634e147dae3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:49.475679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fdf51-777b-4898-98d4-011bd7889cd0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:49.489789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242bfed-1167-401b-ba37-6f37a1fbfe4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:49.491006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f689aa5-fe50-4390-9184-5c5ba1dc9c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:49.491659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f689aa5-fe50-4390-9184-5c5ba1dc9c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:49.491762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242bfed-1167-401b-ba37-6f37a1fbfe4a service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:49.491935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f689aa5-fe50-4390-9184-5c5ba1dc9c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:49.493530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242bfed-1167-401b-ba37-6f37a1fbfe4a service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:49.493964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242bfed-1167-401b-ba37-6f37a1fbfe4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:49.495241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f689aa5-fe50-4390-9184-5c5ba1dc9c92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:49.496188 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f689aa5-fe50-4390-9184-5c5ba1dc9c92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:49.501229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f689aa5-fe50-4390-9184-5c5ba1dc9c92 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:49.575706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 863/1732] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:49 2022] POST /identity/v3/projects => generated 406 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:49.576567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f689aa5-fe50-4390-9184-5c5ba1dc9c92 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:49.706456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242bfed-1167-401b-ba37-6f37a1fbfe4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:49.710394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242bfed-1167-401b-ba37-6f37a1fbfe4a service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:49.743149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 870/1733] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:36:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:36:49.748355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242bfed-1167-401b-ba37-6f37a1fbfe4a service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:49.756600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f689aa5-fe50-4390-9184-5c5ba1dc9c92 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:49.775097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242bfed-1167-401b-ba37-6f37a1fbfe4a service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:49.787692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46275dac-3b0b-4b3a-a9ad-bad74a7f5baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:49.788276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f07860-04ff-4d8d-93d8-be056d374a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:49.789639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f07860-04ff-4d8d-93d8-be056d374a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:49.789639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f07860-04ff-4d8d-93d8-be056d374a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:49.791611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f07860-04ff-4d8d-93d8-be056d374a29 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:49.794061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46275dac-3b0b-4b3a-a9ad-bad74a7f5baa admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:49.794377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46275dac-3b0b-4b3a-a9ad-bad74a7f5baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:49.794649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46275dac-3b0b-4b3a-a9ad-bad74a7f5baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:49.794932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f07860-04ff-4d8d-93d8-be056d374a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:49.795352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46275dac-3b0b-4b3a-a9ad-bad74a7f5baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:49.796990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46275dac-3b0b-4b3a-a9ad-bad74a7f5baa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:49.800548 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f07860-04ff-4d8d-93d8-be056d374a29 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:49.849375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 864/1734] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:49 2022] GET /identity/v3/roles => generated 1427 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:49.859020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46275dac-3b0b-4b3a-a9ad-bad74a7f5baa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:49.892945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46275dac-3b0b-4b3a-a9ad-bad74a7f5baa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:49.903444 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 871/1735] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:49 2022] POST /identity/v3/users => generated 310 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:49.906699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f07860-04ff-4d8d-93d8-be056d374a29 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:49.926886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8094d-7307-4390-97dd-af9a170d187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:49.927601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8094d-7307-4390-97dd-af9a170d187c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:49.927885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8094d-7307-4390-97dd-af9a170d187c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:49.928261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8094d-7307-4390-97dd-af9a170d187c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:49.929170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8094d-7307-4390-97dd-af9a170d187c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:49.930844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8094d-7307-4390-97dd-af9a170d187c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:49.950099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f07860-04ff-4d8d-93d8-be056d374a29 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:49.956092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee30b1-8a03-4c03-93b1-d03c517cf453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:49.956553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee30b1-8a03-4c03-93b1-d03c517cf453 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:49.956842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee30b1-8a03-4c03-93b1-d03c517cf453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:49.957541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee30b1-8a03-4c03-93b1-d03c517cf453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:49.958248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee30b1-8a03-4c03-93b1-d03c517cf453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:49.960025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee30b1-8a03-4c03-93b1-d03c517cf453 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:49.975164 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 872/1736] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:49 2022] GET /identity/v3/roles => generated 1427 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:49.984580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 865/1737] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:49 2022] GET /identity/v3/roles => generated 1427 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:50.001005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8094d-7307-4390-97dd-af9a170d187c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:50.002295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee30b1-8a03-4c03-93b1-d03c517cf453 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:50.023445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8094d-7307-4390-97dd-af9a170d187c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:50.031175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee30b1-8a03-4c03-93b1-d03c517cf453 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:50.047709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64d2ba-b421-49bf-bb2d-3590e8ab3c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:50.051211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64d2ba-b421-49bf-bb2d-3590e8ab3c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:50.051740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64d2ba-b421-49bf-bb2d-3590e8ab3c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:50.052997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64d2ba-b421-49bf-bb2d-3590e8ab3c3c admin admin] PATH_INFO: `/v3/projects/8d2ec49801d946b09cee75ff514be728/users/6e98987dda994601a34051520439d485/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:50.054682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21487151-555f-48a2-9dee-ac8f2877a8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:50.055370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21487151-555f-48a2-9dee-ac8f2877a8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:50.055637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21487151-555f-48a2-9dee-ac8f2877a8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:50.055911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21487151-555f-48a2-9dee-ac8f2877a8b7 admin admin] PATH_INFO: `/v3/projects/00f54a84db95427f930866d6b91816b6/users/0ca879301adb4d0cb6d0f1d7cb8fc308/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:50.095304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64d2ba-b421-49bf-bb2d-3590e8ab3c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2ec49801d946b09cee75ff514be728, user_id=6e98987dda994601a34051520439d485, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:50.099355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64d2ba-b421-49bf-bb2d-3590e8ab3c3c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:50.119820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21487151-555f-48a2-9dee-ac8f2877a8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f54a84db95427f930866d6b91816b6, user_id=0ca879301adb4d0cb6d0f1d7cb8fc308, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:50.123018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21487151-555f-48a2-9dee-ac8f2877a8b7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:50.135638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 873/1738] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:49 2022] PUT /identity/v3/projects/8d2ec49801d946b09cee75ff514be728/users/6e98987dda994601a34051520439d485/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:50.139272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64d2ba-b421-49bf-bb2d-3590e8ab3c3c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:50.181513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 866/1739] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:49 2022] PUT /identity/v3/projects/00f54a84db95427f930866d6b91816b6/users/0ca879301adb4d0cb6d0f1d7cb8fc308/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:50.193432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f2fec-94c0-444b-9794-05badc808185 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:50.193771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f2fec-94c0-444b-9794-05badc808185 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:50.194049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f2fec-94c0-444b-9794-05badc808185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:50.238662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:50.334404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-486f2fec-94c0-444b-9794-05badc808185 None None] MFA Rules not processed for user `0ca879301adb4d0cb6d0f1d7cb8fc308`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:50.335735 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-486f2fec-94c0-444b-9794-05badc808185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:50.383756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64d2ba-b421-49bf-bb2d-3590e8ab3c3c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:50.604303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51545718-93fa-4ab7-959a-d3e70657158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:50.608178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51545718-93fa-4ab7-959a-d3e70657158d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:50.608666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51545718-93fa-4ab7-959a-d3e70657158d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:50.610108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51545718-93fa-4ab7-959a-d3e70657158d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:50.618307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51545718-93fa-4ab7-959a-d3e70657158d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:50.622354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51545718-93fa-4ab7-959a-d3e70657158d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:50.651749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 867/1740] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:36:50.657526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f2fec-94c0-444b-9794-05badc808185 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:50.674861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 874/1741] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:50 2022] GET /identity/v3/roles => generated 1427 bytes in 536 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:50.679663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51545718-93fa-4ab7-959a-d3e70657158d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:50.703552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f2fec-94c0-444b-9794-05badc808185 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:50.712582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51545718-93fa-4ab7-959a-d3e70657158d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:50.719010 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16405d0b-e69d-4397-991c-1d1c73be0c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:50.719519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16405d0b-e69d-4397-991c-1d1c73be0c5a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:50.721964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16405d0b-e69d-4397-991c-1d1c73be0c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:50.722841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16405d0b-e69d-4397-991c-1d1c73be0c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:50.724642 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16405d0b-e69d-4397-991c-1d1c73be0c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:50.726028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa143ad-73f1-4d4f-a3ba-becc24f7a4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:50.727039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa143ad-73f1-4d4f-a3ba-becc24f7a4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:50.731251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa143ad-73f1-4d4f-a3ba-becc24f7a4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:50.734490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16405d0b-e69d-4397-991c-1d1c73be0c5a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:50.754307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa143ad-73f1-4d4f-a3ba-becc24f7a4d4 admin admin] PATH_INFO: `/v3/projects/8d2ec49801d946b09cee75ff514be728/users/6e98987dda994601a34051520439d485/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:50.756503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:50.765149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa143ad-73f1-4d4f-a3ba-becc24f7a4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2ec49801d946b09cee75ff514be728, user_id=6e98987dda994601a34051520439d485, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:50.768333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa143ad-73f1-4d4f-a3ba-becc24f7a4d4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:50.817276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 875/1742] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:50 2022] PUT /identity/v3/projects/8d2ec49801d946b09cee75ff514be728/users/6e98987dda994601a34051520439d485/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:50.827942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae7e30-026d-4a1d-87ba-1273a64dd691 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:50.828434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae7e30-026d-4a1d-87ba-1273a64dd691 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:50.828853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae7e30-026d-4a1d-87ba-1273a64dd691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:50.834983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 868/1743] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:50 2022] POST /identity/v3/projects => generated 380 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:50.838445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16405d0b-e69d-4397-991c-1d1c73be0c5a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:50.882791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-10ae7e30-026d-4a1d-87ba-1273a64dd691 None None] MFA Rules not processed for user `6e98987dda994601a34051520439d485`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:50.902419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-10ae7e30-026d-4a1d-87ba-1273a64dd691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:51.108303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16405d0b-e69d-4397-991c-1d1c73be0c5a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:51.118652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3830a5-6f24-4bab-a416-a590cff4e800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:51.119293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3830a5-6f24-4bab-a416-a590cff4e800 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:51.119779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3830a5-6f24-4bab-a416-a590cff4e800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:51.120063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3830a5-6f24-4bab-a416-a590cff4e800 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:51.120911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3830a5-6f24-4bab-a416-a590cff4e800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:51.125134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3830a5-6f24-4bab-a416-a590cff4e800 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:51.230344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 869/1744] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:50 2022] POST /identity/v3/users => generated 291 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:51.238653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3830a5-6f24-4bab-a416-a590cff4e800 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:51.269203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 876/1745] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:50 2022] POST /identity/v3/auth/tokens => generated 3359 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:51.289393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3830a5-6f24-4bab-a416-a590cff4e800 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:51.304113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67a2c2-6117-42bf-a445-3595b285a647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:51.304754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67a2c2-6117-42bf-a445-3595b285a647 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:51.305032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67a2c2-6117-42bf-a445-3595b285a647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:51.305303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67a2c2-6117-42bf-a445-3595b285a647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:51.306065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67a2c2-6117-42bf-a445-3595b285a647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:51.309489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67a2c2-6117-42bf-a445-3595b285a647 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:51.372577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae7e30-026d-4a1d-87ba-1273a64dd691 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:51.374553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-10ae7e30-026d-4a1d-87ba-1273a64dd691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:51.391899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 870/1746] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:51 2022] GET /identity/v3/roles => generated 1427 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:51.399321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67a2c2-6117-42bf-a445-3595b285a647 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:51.442434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67a2c2-6117-42bf-a445-3595b285a647 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:51.454183 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd47c3f-a090-49ab-837a-a9fc89d19f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:51.455054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47c3f-a090-49ab-837a-a9fc89d19f64 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:51.455654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47c3f-a090-49ab-837a-a9fc89d19f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:51.456113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47c3f-a090-49ab-837a-a9fc89d19f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:51.461402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd47c3f-a090-49ab-837a-a9fc89d19f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:51.461402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd47c3f-a090-49ab-837a-a9fc89d19f64 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:51.513880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 871/1747] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:51 2022] GET /identity/v3/roles => generated 1427 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:51.515460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd47c3f-a090-49ab-837a-a9fc89d19f64 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:51.553416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:51.554163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd47c3f-a090-49ab-837a-a9fc89d19f64 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:51.575082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f996db5-f743-4840-b981-9d6900c1904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:51.576465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f996db5-f743-4840-b981-9d6900c1904d admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:51.576750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f996db5-f743-4840-b981-9d6900c1904d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:51.578828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f996db5-f743-4840-b981-9d6900c1904d admin admin] PATH_INFO: `/v3/projects/c61b082b66e1457ba2238be92ca6bb68/users/32400cd1a6ee4caaae9523c0647399c8/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:51.606244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae7e30-026d-4a1d-87ba-1273a64dd691 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:51.617463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956a7b9-b9eb-4f75-9142-6ae2399eebf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:51.622032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956a7b9-b9eb-4f75-9142-6ae2399eebf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:51.623494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956a7b9-b9eb-4f75-9142-6ae2399eebf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:51.623753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956a7b9-b9eb-4f75-9142-6ae2399eebf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:51.637945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f996db5-f743-4840-b981-9d6900c1904d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61b082b66e1457ba2238be92ca6bb68, user_id=32400cd1a6ee4caaae9523c0647399c8, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:51.645148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f996db5-f743-4840-b981-9d6900c1904d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:51.663569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956a7b9-b9eb-4f75-9142-6ae2399eebf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:51.667543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956a7b9-b9eb-4f75-9142-6ae2399eebf4 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:51.694751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 877/1748] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:51 2022] GET /identity/v3/auth/tokens => generated 3359 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:51.739842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 872/1749] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:51 2022] PUT /identity/v3/projects/c61b082b66e1457ba2238be92ca6bb68/users/32400cd1a6ee4caaae9523c0647399c8/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:51.752893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d48f-056a-4cdb-a6f3-27c23fb587eb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:51.753398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d48f-056a-4cdb-a6f3-27c23fb587eb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:51.753937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d48f-056a-4cdb-a6f3-27c23fb587eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:51.839684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2887d48f-056a-4cdb-a6f3-27c23fb587eb None None] MFA Rules not processed for user `32400cd1a6ee4caaae9523c0647399c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:51.853414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2887d48f-056a-4cdb-a6f3-27c23fb587eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:52.071164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f996db5-f743-4840-b981-9d6900c1904d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:52.126653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 878/1750] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:52.128014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec153f-729a-4979-bb31-06034815420c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:52.128288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec153f-729a-4979-bb31-06034815420c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:52.128561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec153f-729a-4979-bb31-06034815420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:52.171323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-30ec153f-729a-4979-bb31-06034815420c None None] MFA Rules not processed for user `bbe180b45f2e49e2abbf6e833ce0d44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:52.172893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-30ec153f-729a-4979-bb31-06034815420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:52.283641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f996db5-f743-4840-b981-9d6900c1904d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:52.298407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:52.300306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:52.300455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:52.300675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] PATH_INFO: `/v3/users/ded1577ffe054ede888ced02f099beeb` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:52.304772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded1577ffe054ede888ced02f099beeb)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:52.311487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:52.311997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 879/1751] 10.222.0.17 () {58 vars in 1019 bytes} [Thu Sep 1 17:36:52 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:52.313234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec153f-729a-4979-bb31-06034815420c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:52.330421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec153f-729a-4979-bb31-06034815420c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:52.336468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0419a-9893-4956-9cd0-08dec349a95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:52.337397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0419a-9893-4956-9cd0-08dec349a95e admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:52.338176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0419a-9893-4956-9cd0-08dec349a95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:52.339178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0419a-9893-4956-9cd0-08dec349a95e admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:52.340189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0419a-9893-4956-9cd0-08dec349a95e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:52.342802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0419a-9893-4956-9cd0-08dec349a95e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:52.391860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 880/1752] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:52 2022] POST /identity/v3/projects => generated 378 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:52.393086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0419a-9893-4956-9cd0-08dec349a95e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:52.412244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:52.438002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded1577ffe054ede888ced02f099beeb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:52.449928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded1577ffe054ede888ced02f099beeb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:52.451375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded1577ffe054ede888ced02f099beeb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:52.477132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded1577ffe054ede888ced02f099beeb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:52.490214 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded1577ffe054ede888ced02f099beeb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:52.501765 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 873/1753] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:52 2022] DELETE /identity/v3/users/ded1577ffe054ede888ced02f099beeb => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:36:52.503348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:52.505605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:52.617189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0419a-9893-4956-9cd0-08dec349a95e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:52.620714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0e6ba-1dab-45a9-9f1b-19e30da2ceff admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:52.630526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e63b99-9804-48a5-b4f3-495fe46e4992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:52.631351 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e63b99-9804-48a5-b4f3-495fe46e4992 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:52.631515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e63b99-9804-48a5-b4f3-495fe46e4992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:52.631893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e63b99-9804-48a5-b4f3-495fe46e4992 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:52.635887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e63b99-9804-48a5-b4f3-495fe46e4992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:52.640519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e63b99-9804-48a5-b4f3-495fe46e4992 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:52.741907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:52.753861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 874/1754] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:52 2022] POST /identity/v3/users => generated 296 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:52.759256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5376644-5b9d-41c3-a5ed-fdba9d14bb29 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:52.760808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5376644-5b9d-41c3-a5ed-fdba9d14bb29 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:52.760808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5376644-5b9d-41c3-a5ed-fdba9d14bb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:52.838263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f5376644-5b9d-41c3-a5ed-fdba9d14bb29 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:52.842615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f5376644-5b9d-41c3-a5ed-fdba9d14bb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:52.918493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 875/1755] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:52.919350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5376644-5b9d-41c3-a5ed-fdba9d14bb29 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:52.933910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5376644-5b9d-41c3-a5ed-fdba9d14bb29 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:52.957625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdf8c3-34ea-4482-aa17-f6418cd2719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:52.958388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdf8c3-34ea-4482-aa17-f6418cd2719f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:52.958747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdf8c3-34ea-4482-aa17-f6418cd2719f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:52.959092 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdf8c3-34ea-4482-aa17-f6418cd2719f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:52.959823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fdf8c3-34ea-4482-aa17-f6418cd2719f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:52.962692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fdf8c3-34ea-4482-aa17-f6418cd2719f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:53.006288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 876/1756] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:52 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:53.009317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60093dfc-bc54-490e-8312-657c83d32d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:53.010044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093dfc-bc54-490e-8312-657c83d32d31 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:53.013700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093dfc-bc54-490e-8312-657c83d32d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:53.014517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093dfc-bc54-490e-8312-657c83d32d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:53.015336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdf8c3-34ea-4482-aa17-f6418cd2719f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:53.024967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-60093dfc-bc54-490e-8312-657c83d32d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:53.025175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e8fdf8c3-34ea-4482-aa17-f6418cd2719f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:53.068770 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60093dfc-bc54-490e-8312-657c83d32d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:53.073139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60093dfc-bc54-490e-8312-657c83d32d31 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:53.084721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdf8c3-34ea-4482-aa17-f6418cd2719f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:53.102253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54ef4f-a04e-4e3b-9382-54cf1f350827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:53.102888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54ef4f-a04e-4e3b-9382-54cf1f350827 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:53.103169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54ef4f-a04e-4e3b-9382-54cf1f350827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:53.103434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54ef4f-a04e-4e3b-9382-54cf1f350827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:53.140463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 881/1757] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:52 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:53.141604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60093dfc-bc54-490e-8312-657c83d32d31 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:53.164840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54ef4f-a04e-4e3b-9382-54cf1f350827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:53.166897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54ef4f-a04e-4e3b-9382-54cf1f350827 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:53.191501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 877/1758] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:53.208126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60093dfc-bc54-490e-8312-657c83d32d31 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:53.238625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698b24c-58ba-4fbc-a117-d4b09af5ee65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:53.255904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698b24c-58ba-4fbc-a117-d4b09af5ee65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:53.255904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698b24c-58ba-4fbc-a117-d4b09af5ee65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:53.255904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698b24c-58ba-4fbc-a117-d4b09af5ee65 admin admin] PATH_INFO: `/v3/projects/4cd8d50ce44b40f6a25f7e098ac458be/users/b2632fa6da5f4e899273f36faad374e3/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:53.321202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698b24c-58ba-4fbc-a117-d4b09af5ee65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd8d50ce44b40f6a25f7e098ac458be, user_id=b2632fa6da5f4e899273f36faad374e3, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:53.345773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698b24c-58ba-4fbc-a117-d4b09af5ee65 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:53.388133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 882/1759] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:53 2022] PUT /identity/v3/projects/4cd8d50ce44b40f6a25f7e098ac458be/users/b2632fa6da5f4e899273f36faad374e3/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:53.390005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54ef4f-a04e-4e3b-9382-54cf1f350827 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:53.406630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54ef4f-a04e-4e3b-9382-54cf1f350827 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:53.682681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18c545-18b3-4eb7-86a5-f555e9109dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:53.684696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18c545-18b3-4eb7-86a5-f555e9109dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:53.690544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18c545-18b3-4eb7-86a5-f555e9109dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:53.691042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18c545-18b3-4eb7-86a5-f555e9109dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:53.696163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18c545-18b3-4eb7-86a5-f555e9109dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:53.724247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18c545-18b3-4eb7-86a5-f555e9109dd1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:53.767740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698b24c-58ba-4fbc-a117-d4b09af5ee65 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:53.769241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 878/1760] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:53 2022] GET /identity/v3/roles => generated 1427 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:53.825683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698b24c-58ba-4fbc-a117-d4b09af5ee65 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:53.854473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bd7a2-688f-4fda-b0a1-a21cda6f756b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:53.857148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bd7a2-688f-4fda-b0a1-a21cda6f756b admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:53.857994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bd7a2-688f-4fda-b0a1-a21cda6f756b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:53.859612 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bd7a2-688f-4fda-b0a1-a21cda6f756b admin admin] PATH_INFO: `/v3/projects/4cd8d50ce44b40f6a25f7e098ac458be/users/b2632fa6da5f4e899273f36faad374e3/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:53.883790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bd7a2-688f-4fda-b0a1-a21cda6f756b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd8d50ce44b40f6a25f7e098ac458be, user_id=b2632fa6da5f4e899273f36faad374e3, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:53.896136 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bd7a2-688f-4fda-b0a1-a21cda6f756b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:53.896614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18c545-18b3-4eb7-86a5-f555e9109dd1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:53.902622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8b18c545-18b3-4eb7-86a5-f555e9109dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:53.927410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:53.951069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18c545-18b3-4eb7-86a5-f555e9109dd1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:53.967616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 883/1761] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:53 2022] PUT /identity/v3/projects/4cd8d50ce44b40f6a25f7e098ac458be/users/b2632fa6da5f4e899273f36faad374e3/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:53.970235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619838-843e-4a99-a845-327175dfca04 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:53.970648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619838-843e-4a99-a845-327175dfca04 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:53.971106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619838-843e-4a99-a845-327175dfca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:54.100275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dd619838-843e-4a99-a845-327175dfca04 None None] MFA Rules not processed for user `b2632fa6da5f4e899273f36faad374e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:54.103660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dd619838-843e-4a99-a845-327175dfca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:54.238307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:54.239021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:54.239245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:54.239521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] PATH_INFO: `/v3/projects/99517d122b434216838fcafc141e1ba2` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:54.241432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=99517d122b434216838fcafc141e1ba2)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:54.245947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:54.311953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 884/1762] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 01 17:36:54.324672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619838-843e-4a99-a845-327175dfca04 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:54.326332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dd619838-843e-4a99-a845-327175dfca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:54.426937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99517d122b434216838fcafc141e1ba2', 'request_id': 'req-04a80313-124d-421b-a025-b9d168cb4c4e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.453394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99517d122b434216838fcafc141e1ba2', 'request_id': 'req-04a80313-124d-421b-a025-b9d168cb4c4e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.467236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99517d122b434216838fcafc141e1ba2', 'request_id': 'req-04a80313-124d-421b-a025-b9d168cb4c4e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.519286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] The token cache is being invalidate because project 99517d122b434216838fcafc141e1ba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:36:54.519638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.521456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.523259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 879/1763] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:36:53 2022] DELETE /identity/v3/projects/99517d122b434216838fcafc141e1ba2 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:36:54.528368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:54.534839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:54.570759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619838-843e-4a99-a845-327175dfca04 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:54.620298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:54.700123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a77e0-49f3-47d2-8c54-1ecb4d983575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:54.703051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a77e0-49f3-47d2-8c54-1ecb4d983575 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:54.703740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a77e0-49f3-47d2-8c54-1ecb4d983575 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:54.704021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a77e0-49f3-47d2-8c54-1ecb4d983575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:54.714283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5d4a77e0-49f3-47d2-8c54-1ecb4d983575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:54.740965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80313-124d-421b-a025-b9d168cb4c4e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:54.746196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:54.746902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:54.747192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:54.747473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] PATH_INFO: `/v3/users/c47467954c4842a6acb4261f2d3544c9` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:54.748696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c47467954c4842a6acb4261f2d3544c9)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:54.750974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:54.832839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:54.848421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c47467954c4842a6acb4261f2d3544c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.866692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c47467954c4842a6acb4261f2d3544c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.868066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c47467954c4842a6acb4261f2d3544c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.895836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c47467954c4842a6acb4261f2d3544c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.914562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c47467954c4842a6acb4261f2d3544c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:54.937325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 880/1764] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:54 2022] DELETE /identity/v3/users/c47467954c4842a6acb4261f2d3544c9 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:54.955724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a77e0-49f3-47d2-8c54-1ecb4d983575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:54.957403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a77e0-49f3-47d2-8c54-1ecb4d983575 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:54.998075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 885/1765] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:36:55.094307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:55.096193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:55.346112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce562376-0f4e-41aa-bfa7-093bbabfd247 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:55.354460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:55.357913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:55.363012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:55.363500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:55.373559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:55.628727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:55.630567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:55.643362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 881/1766] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:56.028719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20f8bf-f3d4-474c-9f02-39b7892848c7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:56.030979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20f8bf-f3d4-474c-9f02-39b7892848c7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:56.031547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20f8bf-f3d4-474c-9f02-39b7892848c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:56.123708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fb20f8bf-f3d4-474c-9f02-39b7892848c7 None None] MFA Rules not processed for user `bcc538bc852f404699d91d17023ae620`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:56.126155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fb20f8bf-f3d4-474c-9f02-39b7892848c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:56.341254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 886/1767] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:56.710815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:56.715409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:56.961998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e035c0-4602-45cb-809a-d7acd578ecce service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:56.968609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:56.969266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:56.969543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:56.969862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] PATH_INFO: `/v3/users/d7667ec51e3b4e5e8c352dcbf2e08e2e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:56.971182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7667ec51e3b4e5e8c352dcbf2e08e2e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:56.973172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:56.989312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:57.141306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73e6d-6ed2-4e5c-9328-9251b833ece3 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:57.141845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73e6d-6ed2-4e5c-9328-9251b833ece3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:57.142044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73e6d-6ed2-4e5c-9328-9251b833ece3 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:57.143903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 887/1768] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:36:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:36:57.154017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7667ec51e3b4e5e8c352dcbf2e08e2e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:57.174243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7667ec51e3b4e5e8c352dcbf2e08e2e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:57.175015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7667ec51e3b4e5e8c352dcbf2e08e2e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:57.218626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7667ec51e3b4e5e8c352dcbf2e08e2e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:57.225419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2f0c1-0d13-4e4f-8e44-3fb6027e6c6c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:57.225999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2f0c1-0d13-4e4f-8e44-3fb6027e6c6c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:57.226448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2f0c1-0d13-4e4f-8e44-3fb6027e6c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:57.259305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bf8f98b5-d20f-4207-990a-9a33c8c55abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7667ec51e3b4e5e8c352dcbf2e08e2e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:57.266597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-80c2f0c1-0d13-4e4f-8e44-3fb6027e6c6c None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:57.269364 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-80c2f0c1-0d13-4e4f-8e44-3fb6027e6c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:57.280484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 882/1769] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:56 2022] DELETE /identity/v3/users/d7667ec51e3b4e5e8c352dcbf2e08e2e => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:57.285473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02601d6-d82d-403b-98c1-468644c9779c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:57.285793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02601d6-d82d-403b-98c1-468644c9779c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:57.286074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02601d6-d82d-403b-98c1-468644c9779c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:57.326105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b02601d6-d82d-403b-98c1-468644c9779c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:57.328079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b02601d6-d82d-403b-98c1-468644c9779c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:57.435608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 883/1770] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:57.487667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 888/1771] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:36:57 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:57.567478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02601d6-d82d-403b-98c1-468644c9779c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:57.569720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b02601d6-d82d-403b-98c1-468644c9779c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:57.859560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02601d6-d82d-403b-98c1-468644c9779c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:57.891582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf730a-eabb-4b27-814f-d62819c80f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:57.893846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf730a-eabb-4b27-814f-d62819c80f5c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:57.895497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf730a-eabb-4b27-814f-d62819c80f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:57.896825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf730a-eabb-4b27-814f-d62819c80f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:57.971239 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf730a-eabb-4b27-814f-d62819c80f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:57.977921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf730a-eabb-4b27-814f-d62819c80f5c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:57.992577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2f0c1-0d13-4e4f-8e44-3fb6027e6c6c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:57.994128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-80c2f0c1-0d13-4e4f-8e44-3fb6027e6c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:58.024945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2f0c1-0d13-4e4f-8e44-3fb6027e6c6c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:58.044646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:58.045307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:58.045606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:58.045889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] PATH_INFO: `/v3/projects/6f27612fe55545439488b921408683db` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:58.047407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f27612fe55545439488b921408683db)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:58.049366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:58.060011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:58.088746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 884/1772] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:58.192350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f27612fe55545439488b921408683db', 'request_id': 'req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:58.214996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f27612fe55545439488b921408683db', 'request_id': 'req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:58.227133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f27612fe55545439488b921408683db', 'request_id': 'req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:58.267218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] The token cache is being invalidate because project 6f27612fe55545439488b921408683db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:36:58.267320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:58.272240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b8673c8-93d4-45a9-b0dd-dbfcf390ee49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:58.277965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 889/1773] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:36:57 2022] DELETE /identity/v3/projects/6f27612fe55545439488b921408683db => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:36:58.284755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329e427-0f4d-4402-9633-4599ec2ecc38 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:58.285485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329e427-0f4d-4402-9633-4599ec2ecc38 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:58.285916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329e427-0f4d-4402-9633-4599ec2ecc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:58.356983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1329e427-0f4d-4402-9633-4599ec2ecc38 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:58.366705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1329e427-0f4d-4402-9633-4599ec2ecc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:58.575514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:58.759637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 885/1774] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:58.761658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8146f-ed4d-49d7-8a13-1e5b57b6297e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:58.762108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8146f-ed4d-49d7-8a13-1e5b57b6297e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:58.762529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8146f-ed4d-49d7-8a13-1e5b57b6297e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:58.802572 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-24c8146f-ed4d-49d7-8a13-1e5b57b6297e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:36:58.804075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-24c8146f-ed4d-49d7-8a13-1e5b57b6297e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:58.836029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 890/1775] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:36:58.837000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329e427-0f4d-4402-9633-4599ec2ecc38 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:58.858535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329e427-0f4d-4402-9633-4599ec2ecc38 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:58.865401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091fbbd-d6e7-4c6c-a156-de2507935798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:58.867234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091fbbd-d6e7-4c6c-a156-de2507935798 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:58.867701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091fbbd-d6e7-4c6c-a156-de2507935798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:58.868098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091fbbd-d6e7-4c6c-a156-de2507935798 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:58.869082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091fbbd-d6e7-4c6c-a156-de2507935798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:36:58.869710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091fbbd-d6e7-4c6c-a156-de2507935798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:58.872962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091fbbd-d6e7-4c6c-a156-de2507935798 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:58.894845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 886/1776] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:36:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:36:58.899163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8146f-ed4d-49d7-8a13-1e5b57b6297e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:58.917658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8146f-ed4d-49d7-8a13-1e5b57b6297e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:58.927960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c1509-18a5-4d59-8584-142f5f32714b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:58.928999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c1509-18a5-4d59-8584-142f5f32714b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:58.929456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c1509-18a5-4d59-8584-142f5f32714b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:58.929952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c1509-18a5-4d59-8584-142f5f32714b admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:58.930750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c1509-18a5-4d59-8584-142f5f32714b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:58.936492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c1509-18a5-4d59-8584-142f5f32714b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:58.975359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 891/1777] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:36:58 2022] POST /identity/v3/projects => generated 376 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:36:58.979921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091fbbd-d6e7-4c6c-a156-de2507935798 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:58.995208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091fbbd-d6e7-4c6c-a156-de2507935798 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:59.019074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c1509-18a5-4d59-8584-142f5f32714b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:59.020813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c21c1509-18a5-4d59-8584-142f5f32714b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:59.155501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc6f23-4eb2-497b-b5f0-69bea45d9c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:59.156339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc6f23-4eb2-497b-b5f0-69bea45d9c17 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:59.156761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc6f23-4eb2-497b-b5f0-69bea45d9c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:59.157076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc6f23-4eb2-497b-b5f0-69bea45d9c17 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:59.157839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc6f23-4eb2-497b-b5f0-69bea45d9c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:59.160018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc6f23-4eb2-497b-b5f0-69bea45d9c17 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:59.207156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 887/1778] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:36:58 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:36:59.208885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc6f23-4eb2-497b-b5f0-69bea45d9c17 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:59.210690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1abc6f23-4eb2-497b-b5f0-69bea45d9c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:59.238961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc6f23-4eb2-497b-b5f0-69bea45d9c17 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:59.242204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c1509-18a5-4d59-8584-142f5f32714b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:59.248277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:59.249049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:59.249433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:59.249930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] PATH_INFO: `/v3/projects/6ad6aff83c6b40938f933abc30b6c286` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:59.253777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ad6aff83c6b40938f933abc30b6c286)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:59.257973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:59.274045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb2158-9796-4957-bbca-4af391564c90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:59.280654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb2158-9796-4957-bbca-4af391564c90 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:59.280942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb2158-9796-4957-bbca-4af391564c90 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:59.281220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb2158-9796-4957-bbca-4af391564c90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:59.294763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9edb2158-9796-4957-bbca-4af391564c90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:59.345633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb2158-9796-4957-bbca-4af391564c90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:59.347250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb2158-9796-4957-bbca-4af391564c90 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:59.376875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ad6aff83c6b40938f933abc30b6c286', 'request_id': 'req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.390618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ad6aff83c6b40938f933abc30b6c286', 'request_id': 'req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.405161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ad6aff83c6b40938f933abc30b6c286', 'request_id': 'req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.419012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 892/1779] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:36:59 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:36:59.420014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb2158-9796-4957-bbca-4af391564c90 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:59.439733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] The token cache is being invalidate because project 6ad6aff83c6b40938f933abc30b6c286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:36:59.439949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.446808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.455399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 888/1780] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:36:59 2022] DELETE /identity/v3/projects/6ad6aff83c6b40938f933abc30b6c286 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:59.456850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:59.461934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:59.489035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb2158-9796-4957-bbca-4af391564c90 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:59.554564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:59.618189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:36:59.627864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22584ad-b1ec-404e-b84a-9e2fbfc01f5f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:59.643133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:59.644020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:59.644417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:59.644793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] PATH_INFO: `/v3/users/6e98987dda994601a34051520439d485` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:59.649451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e98987dda994601a34051520439d485)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:59.668435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59364e-52e6-40dd-8d35-4e4e35b753fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:59.671228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59364e-52e6-40dd-8d35-4e4e35b753fb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:59.674193 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59364e-52e6-40dd-8d35-4e4e35b753fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:59.675524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59364e-52e6-40dd-8d35-4e4e35b753fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:59.676235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59364e-52e6-40dd-8d35-4e4e35b753fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:59.676389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:59.684155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59364e-52e6-40dd-8d35-4e4e35b753fb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:59.738213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 893/1781] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:59 2022] GET /identity/v3/roles => generated 1427 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:59.743948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59364e-52e6-40dd-8d35-4e4e35b753fb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:59.748638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bb59364e-52e6-40dd-8d35-4e4e35b753fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:59.782378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59364e-52e6-40dd-8d35-4e4e35b753fb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:59.805450 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa26f9-6a23-405b-9f93-fd1f1cefa417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:59.812186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa26f9-6a23-405b-9f93-fd1f1cefa417 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:59.812553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa26f9-6a23-405b-9f93-fd1f1cefa417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:59.812840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa26f9-6a23-405b-9f93-fd1f1cefa417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:59.819034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa26f9-6a23-405b-9f93-fd1f1cefa417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:59.823623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa26f9-6a23-405b-9f93-fd1f1cefa417 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:59.833380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e98987dda994601a34051520439d485'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.848933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e98987dda994601a34051520439d485'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.850148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e98987dda994601a34051520439d485'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.857621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 894/1782] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:36:59 2022] GET /identity/v3/roles => generated 1427 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:36:59.860203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa26f9-6a23-405b-9f93-fd1f1cefa417 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:59.882857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa26f9-6a23-405b-9f93-fd1f1cefa417 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:36:59.884619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e98987dda994601a34051520439d485'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.897441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927d66e-ccd9-4a88-9324-7c25b4b0b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:36:59.898382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927d66e-ccd9-4a88-9324-7c25b4b0b223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:59.898382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927d66e-ccd9-4a88-9324-7c25b4b0b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:59.898667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927d66e-ccd9-4a88-9324-7c25b4b0b223 admin admin] PATH_INFO: `/v3/projects/04c8ea419f39467690de6c5f6056cebe/users/982d0778ec684600a315b0cda8c47a8a/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:36:59.900295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e98987dda994601a34051520439d485'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:36:59.917622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 889/1783] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:59 2022] DELETE /identity/v3/users/6e98987dda994601a34051520439d485 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:59.922939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927d66e-ccd9-4a88-9324-7c25b4b0b223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c8ea419f39467690de6c5f6056cebe, user_id=982d0778ec684600a315b0cda8c47a8a, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:36:59.925621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927d66e-ccd9-4a88-9324-7c25b4b0b223 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:36:59.957360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:36:59.958938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:36:59.960260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 895/1784] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:36:59 2022] PUT /identity/v3/projects/04c8ea419f39467690de6c5f6056cebe/users/982d0778ec684600a315b0cda8c47a8a/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:36:59.967881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e4cb-8e7e-4064-b137-35df55dec1b2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:36:59.968179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e4cb-8e7e-4064-b137-35df55dec1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:36:59.968447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e4cb-8e7e-4064-b137-35df55dec1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:00.037336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f997e4cb-8e7e-4064-b137-35df55dec1b2 None None] MFA Rules not processed for user `982d0778ec684600a315b0cda8c47a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:00.040002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f997e4cb-8e7e-4064-b137-35df55dec1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:00.100803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e912c2-e29a-4ad0-8b32-0532e0895fb7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:00.206672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 896/1785] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:36:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:00.208282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a60846-f828-4f1d-84fc-7d8006d1d580 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:00.208580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a60846-f828-4f1d-84fc-7d8006d1d580 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:00.208857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a60846-f828-4f1d-84fc-7d8006d1d580 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:00.212089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 897/1786] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:37:00 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:37:00.215519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a60846-f828-4f1d-84fc-7d8006d1d580 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:00.219637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-59a60846-f828-4f1d-84fc-7d8006d1d580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:00.231784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:00.248135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:00.249797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:00.250657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:00.250942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] PATH_INFO: `/v3/users/0ca879301adb4d0cb6d0f1d7cb8fc308` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:00.262132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca879301adb4d0cb6d0f1d7cb8fc308)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:00.262132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:00.378916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca879301adb4d0cb6d0f1d7cb8fc308'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:00.395848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca879301adb4d0cb6d0f1d7cb8fc308'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:00.396548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca879301adb4d0cb6d0f1d7cb8fc308'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:00.407832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a60846-f828-4f1d-84fc-7d8006d1d580 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:00.420112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e347e9e-8438-4237-a077-15b5f5316c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:00.420765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e347e9e-8438-4237-a077-15b5f5316c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:00.421303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e347e9e-8438-4237-a077-15b5f5316c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:00.421613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e347e9e-8438-4237-a077-15b5f5316c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:00.423496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e347e9e-8438-4237-a077-15b5f5316c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:00.426750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca879301adb4d0cb6d0f1d7cb8fc308'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:00.429260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e347e9e-8438-4237-a077-15b5f5316c99 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:00.443992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3484639f-ede1-4b4d-b3f0-34a36fde34ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca879301adb4d0cb6d0f1d7cb8fc308'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:00.461242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 890/1787] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:36:59 2022] DELETE /identity/v3/users/0ca879301adb4d0cb6d0f1d7cb8fc308 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:00.466806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712982f2-bc15-4be0-bbe4-5b3e3e703690 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:00.467097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712982f2-bc15-4be0-bbe4-5b3e3e703690 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:00.467377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712982f2-bc15-4be0-bbe4-5b3e3e703690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:00.492859 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 898/1788] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:00 2022] POST /identity/v3/projects => generated 376 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:00.501509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e354ad-8437-4103-b1c5-092a36041042 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:00.501840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e354ad-8437-4103-b1c5-092a36041042 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:00.503179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e354ad-8437-4103-b1c5-092a36041042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:00.520216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-712982f2-bc15-4be0-bbe4-5b3e3e703690 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:00.524813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-712982f2-bc15-4be0-bbe4-5b3e3e703690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:00.544799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-03e354ad-8437-4103-b1c5-092a36041042 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:00.550087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-03e354ad-8437-4103-b1c5-092a36041042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:00.612061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:00.720192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 891/1789] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:37:00 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:00.724910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712982f2-bc15-4be0-bbe4-5b3e3e703690 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:00.730809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-712982f2-bc15-4be0-bbe4-5b3e3e703690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:00.787463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 899/1790] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:00.792732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e354ad-8437-4103-b1c5-092a36041042 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:00.797561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-03e354ad-8437-4103-b1c5-092a36041042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:00.861343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e354ad-8437-4103-b1c5-092a36041042 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:00.874918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45e2fb-1f99-4994-98b0-09c9157cb13f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:00.877199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e2fb-1f99-4994-98b0-09c9157cb13f service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:00.877675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e2fb-1f99-4994-98b0-09c9157cb13f service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:00.879465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e2fb-1f99-4994-98b0-09c9157cb13f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:00.905359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45e2fb-1f99-4994-98b0-09c9157cb13f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:00.920242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45e2fb-1f99-4994-98b0-09c9157cb13f service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:00.925905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712982f2-bc15-4be0-bbe4-5b3e3e703690 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:00.934294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b224b2-bbd8-4d6f-a23a-69ac604e23b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:00.934851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b224b2-bbd8-4d6f-a23a-69ac604e23b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:00.935241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b224b2-bbd8-4d6f-a23a-69ac604e23b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:00.935819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b224b2-bbd8-4d6f-a23a-69ac604e23b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:00.936801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b224b2-bbd8-4d6f-a23a-69ac604e23b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:00.939245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b224b2-bbd8-4d6f-a23a-69ac604e23b9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:00.973565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 900/1791] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:00 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:01.012221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:01.035141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 892/1792] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:00 2022] POST /identity/v3/users => generated 295 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:01.040293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45e2fb-1f99-4994-98b0-09c9157cb13f service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:01.082441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45e2fb-1f99-4994-98b0-09c9157cb13f service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:01.105144 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f01c75-7242-43d2-8b3a-82b1cfafb1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:01.105821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f01c75-7242-43d2-8b3a-82b1cfafb1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:01.106108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f01c75-7242-43d2-8b3a-82b1cfafb1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:01.108605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f01c75-7242-43d2-8b3a-82b1cfafb1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:01.111187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f01c75-7242-43d2-8b3a-82b1cfafb1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:01.114349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f01c75-7242-43d2-8b3a-82b1cfafb1b0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:01.190045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 901/1793] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:01 2022] GET /identity/v3/roles => generated 1427 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:01.190513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b224b2-bbd8-4d6f-a23a-69ac604e23b9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:01.206600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b224b2-bbd8-4d6f-a23a-69ac604e23b9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:01.222034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406fb17-0a3d-4195-895c-3763433d1c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:01.222829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406fb17-0a3d-4195-895c-3763433d1c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:01.223185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406fb17-0a3d-4195-895c-3763433d1c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:01.223532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406fb17-0a3d-4195-895c-3763433d1c1e admin admin] PATH_INFO: `/v3/projects/654f9f4c047142deb95bf60f01f3f0fd/users/34e04c5c531f4890b4bffeee35cfc4da/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:01.257285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406fb17-0a3d-4195-895c-3763433d1c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=654f9f4c047142deb95bf60f01f3f0fd, user_id=34e04c5c531f4890b4bffeee35cfc4da, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:01.260895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406fb17-0a3d-4195-895c-3763433d1c1e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:01.292689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 893/1794] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:01 2022] PUT /identity/v3/projects/654f9f4c047142deb95bf60f01f3f0fd/users/34e04c5c531f4890b4bffeee35cfc4da/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:01.297216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f01c75-7242-43d2-8b3a-82b1cfafb1b0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:01.319690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f01c75-7242-43d2-8b3a-82b1cfafb1b0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:01.434604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406fb17-0a3d-4195-895c-3763433d1c1e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:01.436605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5406fb17-0a3d-4195-895c-3763433d1c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:01.537645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e30ca-e434-44e9-9d84-c4ac3dde9483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:01.538280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e30ca-e434-44e9-9d84-c4ac3dde9483 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:01.538684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e30ca-e434-44e9-9d84-c4ac3dde9483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:01.539025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e30ca-e434-44e9-9d84-c4ac3dde9483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:01.549900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e30ca-e434-44e9-9d84-c4ac3dde9483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:01.552615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e30ca-e434-44e9-9d84-c4ac3dde9483 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:01.574383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 902/1795] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:01 2022] GET /identity/v3/roles => generated 1427 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:01.577837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e30ca-e434-44e9-9d84-c4ac3dde9483 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:01.606674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406fb17-0a3d-4195-895c-3763433d1c1e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:01.617426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e30ca-e434-44e9-9d84-c4ac3dde9483 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:01.625306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:01.628130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:01.629776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:01.630138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] PATH_INFO: `/v3/projects/8d2ec49801d946b09cee75ff514be728` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:01.634452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bb4c7-cc06-4f7e-8142-83b31ca7030d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:01.637672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d2ec49801d946b09cee75ff514be728)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:01.638885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bb4c7-cc06-4f7e-8142-83b31ca7030d admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:01.639265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bb4c7-cc06-4f7e-8142-83b31ca7030d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:01.639577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bb4c7-cc06-4f7e-8142-83b31ca7030d admin admin] PATH_INFO: `/v3/projects/654f9f4c047142deb95bf60f01f3f0fd/users/34e04c5c531f4890b4bffeee35cfc4da/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:01.641163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:01.648497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bb4c7-cc06-4f7e-8142-83b31ca7030d admin admin] RBAC: Authorizing `identity:create_grant(project_id=654f9f4c047142deb95bf60f01f3f0fd, user_id=34e04c5c531f4890b4bffeee35cfc4da, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:01.652319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bb4c7-cc06-4f7e-8142-83b31ca7030d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:01.703897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 903/1796] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:01 2022] PUT /identity/v3/projects/654f9f4c047142deb95bf60f01f3f0fd/users/34e04c5c531f4890b4bffeee35cfc4da/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:01.717073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacac523-7e50-4683-a236-07e1698758f3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:01.717366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacac523-7e50-4683-a236-07e1698758f3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:01.717800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacac523-7e50-4683-a236-07e1698758f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:01.832266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d2ec49801d946b09cee75ff514be728', 'request_id': 'req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:01.836392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bacac523-7e50-4683-a236-07e1698758f3 None None] MFA Rules not processed for user `34e04c5c531f4890b4bffeee35cfc4da`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:01.841332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bacac523-7e50-4683-a236-07e1698758f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:01.850954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d2ec49801d946b09cee75ff514be728', 'request_id': 'req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:01.871184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d2ec49801d946b09cee75ff514be728', 'request_id': 'req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:01.905878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] The token cache is being invalidate because project 8d2ec49801d946b09cee75ff514be728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:01.914751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:01.915451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:01.917993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 894/1797] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:01 2022] DELETE /identity/v3/projects/8d2ec49801d946b09cee75ff514be728 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:01.918834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:01.920198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:02.013181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e5ff4-df2a-4cee-b7b6-a141ee6d1aff admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:02.020729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:02.024261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.024968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.025677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.028065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:02.053072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:02.108260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 904/1798] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:02.110381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff68766-1f8b-43e6-aa11-76b553ae12d3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.112223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff68766-1f8b-43e6-aa11-76b553ae12d3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.113365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff68766-1f8b-43e6-aa11-76b553ae12d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.162664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3ff68766-1f8b-43e6-aa11-76b553ae12d3 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:02.172473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3ff68766-1f8b-43e6-aa11-76b553ae12d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:02.181331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:02.227371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:02.231529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:02.256622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 895/1799] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:02.259812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:02.263667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:02.345948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 905/1800] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:02.347363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff68766-1f8b-43e6-aa11-76b553ae12d3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:02.350901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3ff68766-1f8b-43e6-aa11-76b553ae12d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:02.355485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a20a9-8fb8-4be1-8594-33dfd2de478c service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:02.360129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06531e87-1380-47a5-8957-847d0e394bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:02.361109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06531e87-1380-47a5-8957-847d0e394bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.361752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06531e87-1380-47a5-8957-847d0e394bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.362232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06531e87-1380-47a5-8957-847d0e394bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.364447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-06531e87-1380-47a5-8957-847d0e394bb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:02.373694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff68766-1f8b-43e6-aa11-76b553ae12d3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:02.380043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:02.380725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.381019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.381298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] PATH_INFO: `/v3/projects/dba6ce0b2305423790e5153ee702f1c4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.383380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba6ce0b2305423790e5153ee702f1c4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:02.386048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:02.480088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba6ce0b2305423790e5153ee702f1c4', 'request_id': 'req-96fad599-64d6-4969-b85e-204339eb71c9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.488019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba6ce0b2305423790e5153ee702f1c4', 'request_id': 'req-96fad599-64d6-4969-b85e-204339eb71c9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.491630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06531e87-1380-47a5-8957-847d0e394bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:02.495814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba6ce0b2305423790e5153ee702f1c4', 'request_id': 'req-96fad599-64d6-4969-b85e-204339eb71c9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.496623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06531e87-1380-47a5-8957-847d0e394bb9 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:02.512984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] The token cache is being invalidate because project dba6ce0b2305423790e5153ee702f1c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:02.513501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.514314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96fad599-64d6-4969-b85e-204339eb71c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.516283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 906/1801] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:02 2022] DELETE /identity/v3/projects/dba6ce0b2305423790e5153ee702f1c4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:02.521010 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 896/1802] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:02.522823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06531e87-1380-47a5-8957-847d0e394bb9 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:02.531973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd9c5c-02a7-49d1-9923-889f7581b9a0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.533619 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd9c5c-02a7-49d1-9923-889f7581b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.535425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd9c5c-02a7-49d1-9923-889f7581b9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.549843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06531e87-1380-47a5-8957-847d0e394bb9 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:02.567622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-55dd9c5c-02a7-49d1-9923-889f7581b9a0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:02.569323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-55dd9c5c-02a7-49d1-9923-889f7581b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:02.667779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:02.668804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.669293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.669804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] PATH_INFO: `/v3/users/0ab7f9049c2a4511817c7e47107a9c87` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.671469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ab7f9049c2a4511817c7e47107a9c87)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:02.673653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:02.683855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 907/1803] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:02.686437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:02.687728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd9c5c-02a7-49d1-9923-889f7581b9a0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:02.697893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd9c5c-02a7-49d1-9923-889f7581b9a0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:02.703063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629df24-e374-4c50-b2bc-f41332cb5a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:02.703682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629df24-e374-4c50-b2bc-f41332cb5a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.703967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629df24-e374-4c50-b2bc-f41332cb5a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.704267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629df24-e374-4c50-b2bc-f41332cb5a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.704917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629df24-e374-4c50-b2bc-f41332cb5a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:02.705222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629df24-e374-4c50-b2bc-f41332cb5a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:02.707083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629df24-e374-4c50-b2bc-f41332cb5a90 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:02.724138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 908/1804] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:02.725936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629df24-e374-4c50-b2bc-f41332cb5a90 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:02.742871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629df24-e374-4c50-b2bc-f41332cb5a90 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:02.743601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ab7f9049c2a4511817c7e47107a9c87'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.752078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f7495-3b6e-4d6b-b592-bc8711f10141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:02.752704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f7495-3b6e-4d6b-b592-bc8711f10141 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.753374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f7495-3b6e-4d6b-b592-bc8711f10141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.753374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f7495-3b6e-4d6b-b592-bc8711f10141 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.753907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f7495-3b6e-4d6b-b592-bc8711f10141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:02.756351 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ab7f9049c2a4511817c7e47107a9c87'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.756485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f7495-3b6e-4d6b-b592-bc8711f10141 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:02.757598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ab7f9049c2a4511817c7e47107a9c87'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.779409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ab7f9049c2a4511817c7e47107a9c87'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.788204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:02.791123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ab7f9049c2a4511817c7e47107a9c87'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:02.801262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 909/1805] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:02 2022] POST /identity/v3/projects => generated 388 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:02.801262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f7495-3b6e-4d6b-b592-bc8711f10141 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:02.803451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-684f7495-3b6e-4d6b-b592-bc8711f10141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:02.805155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 897/1806] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:02 2022] DELETE /identity/v3/users/0ab7f9049c2a4511817c7e47107a9c87 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:02.806261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:02.809756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:02.912947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4a8be-d19a-49b1-92cc-1d89368a1ba2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:02.925250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e446b1-7f93-45ee-bfb4-e6de5d65547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:02.929623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e446b1-7f93-45ee-bfb4-e6de5d65547f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.929623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e446b1-7f93-45ee-bfb4-e6de5d65547f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.929623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e446b1-7f93-45ee-bfb4-e6de5d65547f admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.930918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e446b1-7f93-45ee-bfb4-e6de5d65547f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:02.932923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e446b1-7f93-45ee-bfb4-e6de5d65547f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:02.966702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f7495-3b6e-4d6b-b592-bc8711f10141 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:02.977989 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:02.978846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:02.979173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:02.979454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] PATH_INFO: `/v3/projects/00f54a84db95427f930866d6b91816b6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:02.985518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 898/1807] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:02 2022] POST /identity/v3/users => generated 295 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:02.985708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f54a84db95427f930866d6b91816b6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:02.985708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:02.986980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e446b1-7f93-45ee-bfb4-e6de5d65547f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.001881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e446b1-7f93-45ee-bfb4-e6de5d65547f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.024050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95242062-6dfd-4437-844c-43c8e795f88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.025338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95242062-6dfd-4437-844c-43c8e795f88b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.040511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95242062-6dfd-4437-844c-43c8e795f88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.041031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95242062-6dfd-4437-844c-43c8e795f88b admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.046169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95242062-6dfd-4437-844c-43c8e795f88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.051426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95242062-6dfd-4437-844c-43c8e795f88b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:03.069625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 899/1808] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:02 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:03.071394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95242062-6dfd-4437-844c-43c8e795f88b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.083569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95242062-6dfd-4437-844c-43c8e795f88b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.103109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f54a84db95427f930866d6b91816b6', 'request_id': 'req-830cc3de-2328-495f-9702-3bdf1cf9631b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.112805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f54a84db95427f930866d6b91816b6', 'request_id': 'req-830cc3de-2328-495f-9702-3bdf1cf9631b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.130492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f54a84db95427f930866d6b91816b6', 'request_id': 'req-830cc3de-2328-495f-9702-3bdf1cf9631b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.145710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] The token cache is being invalidate because project 00f54a84db95427f930866d6b91816b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:03.146446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.147505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.149625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 910/1809] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:02 2022] DELETE /identity/v3/projects/00f54a84db95427f930866d6b91816b6 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:03.153850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.156152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:03.173192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d388c2-b58e-4bc4-851a-ee5fbb87eeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.174040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d388c2-b58e-4bc4-851a-ee5fbb87eeb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.174422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d388c2-b58e-4bc4-851a-ee5fbb87eeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.178407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d388c2-b58e-4bc4-851a-ee5fbb87eeb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.179125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d388c2-b58e-4bc4-851a-ee5fbb87eeb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.180832 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d388c2-b58e-4bc4-851a-ee5fbb87eeb8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:03.196813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 900/1810] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:03 2022] GET /identity/v3/roles => generated 1427 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:03.198777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d388c2-b58e-4bc4-851a-ee5fbb87eeb8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.202483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a2d388c2-b58e-4bc4-851a-ee5fbb87eeb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:03.211664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:03.218947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d388c2-b58e-4bc4-851a-ee5fbb87eeb8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.225353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.226407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.226717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.226969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] PATH_INFO: `/v3/projects/efb42e99c1a84667a09e9cd1af4f8d4b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.228843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb42e99c1a84667a09e9cd1af4f8d4b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.231814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:03.254191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc3de-2328-495f-9702-3bdf1cf9631b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.259390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.260687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.261045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.261385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] PATH_INFO: `/v3/users/32400cd1a6ee4caaae9523c0647399c8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.262691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32400cd1a6ee4caaae9523c0647399c8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.264628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:03.292207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb42e99c1a84667a09e9cd1af4f8d4b', 'request_id': 'req-83f39aca-cb8d-4ef3-952c-660063cac924'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.302940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb42e99c1a84667a09e9cd1af4f8d4b', 'request_id': 'req-83f39aca-cb8d-4ef3-952c-660063cac924'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.305533 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:03.312003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb42e99c1a84667a09e9cd1af4f8d4b', 'request_id': 'req-83f39aca-cb8d-4ef3-952c-660063cac924'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.325420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32400cd1a6ee4caaae9523c0647399c8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.327800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] The token cache is being invalidate because project efb42e99c1a84667a09e9cd1af4f8d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:03.328126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.328758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.330964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 901/1811] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:03 2022] DELETE /identity/v3/projects/efb42e99c1a84667a09e9cd1af4f8d4b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:03.331718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.333076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:03.340113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32400cd1a6ee4caaae9523c0647399c8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.347675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32400cd1a6ee4caaae9523c0647399c8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.359879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32400cd1a6ee4caaae9523c0647399c8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.360636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3851e940-b575-402c-978a-5dd6c841aa14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32400cd1a6ee4caaae9523c0647399c8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.368100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 911/1812] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:03 2022] DELETE /identity/v3/users/32400cd1a6ee4caaae9523c0647399c8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:03.377814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2df234-d70a-44af-977b-9a8d938d438e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.378302 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2df234-d70a-44af-977b-9a8d938d438e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.378572 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2df234-d70a-44af-977b-9a8d938d438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.405380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1a2df234-d70a-44af-977b-9a8d938d438e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:03.406774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1a2df234-d70a-44af-977b-9a8d938d438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:03.421088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39aca-cb8d-4ef3-952c-660063cac924 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.479502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 912/1813] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:03.491567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f2388-a7f0-49cb-938a-f677aa6e3a59 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.491879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f2388-a7f0-49cb-938a-f677aa6e3a59 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.492176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f2388-a7f0-49cb-938a-f677aa6e3a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.515951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ecf9c-a6d0-4dd3-a7e2-31fe971d5fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.518458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ecf9c-a6d0-4dd3-a7e2-31fe971d5fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.518965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ecf9c-a6d0-4dd3-a7e2-31fe971d5fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.519457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ecf9c-a6d0-4dd3-a7e2-31fe971d5fc4 admin admin] PATH_INFO: `/v3/projects/346083745ed84ee198df340431a6bc7a/users/d5bd874497264bba806d407b655cf7e4/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.522377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-178f2388-a7f0-49cb-938a-f677aa6e3a59 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:03.524185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-178f2388-a7f0-49cb-938a-f677aa6e3a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:03.538754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 913/1814] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:03.540773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f2388-a7f0-49cb-938a-f677aa6e3a59 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.554984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f2388-a7f0-49cb-938a-f677aa6e3a59 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.561727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5754445-1d02-453d-a7e6-bd1b13de6759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.562473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5754445-1d02-453d-a7e6-bd1b13de6759 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.562761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5754445-1d02-453d-a7e6-bd1b13de6759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.563036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5754445-1d02-453d-a7e6-bd1b13de6759 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.563501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ecf9c-a6d0-4dd3-a7e2-31fe971d5fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=346083745ed84ee198df340431a6bc7a, user_id=d5bd874497264bba806d407b655cf7e4, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.563752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5754445-1d02-453d-a7e6-bd1b13de6759 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:03.564109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5754445-1d02-453d-a7e6-bd1b13de6759 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.565778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5754445-1d02-453d-a7e6-bd1b13de6759 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:03.570158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ecf9c-a6d0-4dd3-a7e2-31fe971d5fc4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:03.588257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 914/1815] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:03.589484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5754445-1d02-453d-a7e6-bd1b13de6759 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.601354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 902/1816] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:03 2022] PUT /identity/v3/projects/346083745ed84ee198df340431a6bc7a/users/d5bd874497264bba806d407b655cf7e4/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:03.602960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5754445-1d02-453d-a7e6-bd1b13de6759 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.613420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23a58d-87f1-4710-97cc-bcc632aaf8c3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.613741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23a58d-87f1-4710-97cc-bcc632aaf8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.614068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23a58d-87f1-4710-97cc-bcc632aaf8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.671657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3f23a58d-87f1-4710-97cc-bcc632aaf8c3 None None] MFA Rules not processed for user `d5bd874497264bba806d407b655cf7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:03.674547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3f23a58d-87f1-4710-97cc-bcc632aaf8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:03.686200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:03.705172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0247ff-487a-43a9-8a8d-156c233b2206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.710384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0247ff-487a-43a9-8a8d-156c233b2206 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.710677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0247ff-487a-43a9-8a8d-156c233b2206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.710961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0247ff-487a-43a9-8a8d-156c233b2206 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.711615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0247ff-487a-43a9-8a8d-156c233b2206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.713721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0247ff-487a-43a9-8a8d-156c233b2206 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:03.753743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 915/1817] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:03 2022] POST /identity/v3/projects => generated 366 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:03.753743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0247ff-487a-43a9-8a8d-156c233b2206 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.756344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c0247ff-487a-43a9-8a8d-156c233b2206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:03.773560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:03.809007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 903/1818] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:03 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:03.809007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23a58d-87f1-4710-97cc-bcc632aaf8c3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.817881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23a58d-87f1-4710-97cc-bcc632aaf8c3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.849846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0247ff-487a-43a9-8a8d-156c233b2206 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.856190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.856190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.856455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.856634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] PATH_INFO: `/v3/projects/c61b082b66e1457ba2238be92ca6bb68` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.857799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61b082b66e1457ba2238be92ca6bb68)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.859810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:03.878638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d08c35-5ff5-403d-9b6b-5d76bf81ed1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.879243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d08c35-5ff5-403d-9b6b-5d76bf81ed1b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.879522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d08c35-5ff5-403d-9b6b-5d76bf81ed1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.879797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d08c35-5ff5-403d-9b6b-5d76bf81ed1b admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.880420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d08c35-5ff5-403d-9b6b-5d76bf81ed1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.882361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d08c35-5ff5-403d-9b6b-5d76bf81ed1b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:03.905902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 904/1819] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:03 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:03.907115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d08c35-5ff5-403d-9b6b-5d76bf81ed1b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.908789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-58d08c35-5ff5-403d-9b6b-5d76bf81ed1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:03.912787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61b082b66e1457ba2238be92ca6bb68', 'request_id': 'req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.922233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61b082b66e1457ba2238be92ca6bb68', 'request_id': 'req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.922384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d08c35-5ff5-403d-9b6b-5d76bf81ed1b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:03.928598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61b082b66e1457ba2238be92ca6bb68', 'request_id': 'req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.947632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] The token cache is being invalidate because project c61b082b66e1457ba2238be92ca6bb68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:03.947930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.948634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:03.950271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 916/1820] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:03 2022] DELETE /identity/v3/projects/c61b082b66e1457ba2238be92ca6bb68 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:03.954374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:03.954374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:03.984684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b21d9-b46c-4330-948b-4fd65b0e1c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:03.985454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b21d9-b46c-4330-948b-4fd65b0e1c24 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:03.985771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b21d9-b46c-4330-948b-4fd65b0e1c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:03.986043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b21d9-b46c-4330-948b-4fd65b0e1c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:03.986672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b21d9-b46c-4330-948b-4fd65b0e1c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:03.988694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b21d9-b46c-4330-948b-4fd65b0e1c24 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.019934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 905/1821] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:03 2022] POST /identity/v3/projects => generated 386 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:04.020763 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b21d9-b46c-4330-948b-4fd65b0e1c24 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.023081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0b7b21d9-b46c-4330-948b-4fd65b0e1c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.031522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a5314-3c0e-45be-b50d-c1fa9eff6f3b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.047866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:04.094126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:04.099813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b21d9-b46c-4330-948b-4fd65b0e1c24 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.105647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.106224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.106503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.106774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] PATH_INFO: `/v3/users/b2632fa6da5f4e899273f36faad374e3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.106908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb15da-b1b1-4446-929c-8d1d8ae5f4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.107635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb15da-b1b1-4446-929c-8d1d8ae5f4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.107911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2632fa6da5f4e899273f36faad374e3)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.108009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb15da-b1b1-4446-929c-8d1d8ae5f4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.108186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb15da-b1b1-4446-929c-8d1d8ae5f4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.108828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb15da-b1b1-4446-929c-8d1d8ae5f4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.109864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.110445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb15da-b1b1-4446-929c-8d1d8ae5f4c5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.120680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 917/1822] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:03 2022] GET /identity/v3/roles => generated 1427 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:04.121688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb15da-b1b1-4446-929c-8d1d8ae5f4c5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.123420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cffb15da-b1b1-4446-929c-8d1d8ae5f4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.136168 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb15da-b1b1-4446-929c-8d1d8ae5f4c5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.141422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b261203-d6ae-4c3c-a353-64144638ea48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.142359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b261203-d6ae-4c3c-a353-64144638ea48 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.142638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b261203-d6ae-4c3c-a353-64144638ea48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.142948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b261203-d6ae-4c3c-a353-64144638ea48 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.144094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b261203-d6ae-4c3c-a353-64144638ea48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.146091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b261203-d6ae-4c3c-a353-64144638ea48 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.158353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2632fa6da5f4e899273f36faad374e3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.166654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2632fa6da5f4e899273f36faad374e3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.167206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2632fa6da5f4e899273f36faad374e3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.173512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 918/1823] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:04 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:04.173512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b261203-d6ae-4c3c-a353-64144638ea48 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.180265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2632fa6da5f4e899273f36faad374e3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.183734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b261203-d6ae-4c3c-a353-64144638ea48 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.186947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2632fa6da5f4e899273f36faad374e3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.188636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3880f2b-cf07-4c23-9d95-0f7344ddc565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.189246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3880f2b-cf07-4c23-9d95-0f7344ddc565 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.189526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3880f2b-cf07-4c23-9d95-0f7344ddc565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.189832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3880f2b-cf07-4c23-9d95-0f7344ddc565 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.190503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3880f2b-cf07-4c23-9d95-0f7344ddc565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.192098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3880f2b-cf07-4c23-9d95-0f7344ddc565 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.193352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 906/1824] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:04 2022] DELETE /identity/v3/users/b2632fa6da5f4e899273f36faad374e3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:04.194135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.198116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.208157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 919/1825] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:04 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:04.209212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3880f2b-cf07-4c23-9d95-0f7344ddc565 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.210951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d3880f2b-cf07-4c23-9d95-0f7344ddc565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.224391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3880f2b-cf07-4c23-9d95-0f7344ddc565 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.286550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4e7d0-0086-4a62-9c31-170e2363eb51 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.292131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97d260-7f66-4245-98bc-24133dd7b622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.292543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97d260-7f66-4245-98bc-24133dd7b622 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.292821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97d260-7f66-4245-98bc-24133dd7b622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.293103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97d260-7f66-4245-98bc-24133dd7b622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.293769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97d260-7f66-4245-98bc-24133dd7b622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.295373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97d260-7f66-4245-98bc-24133dd7b622 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.297316 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947d5a2-c68e-4c63-8d17-4c92729667d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.298598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947d5a2-c68e-4c63-8d17-4c92729667d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.299119 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947d5a2-c68e-4c63-8d17-4c92729667d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.299628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947d5a2-c68e-4c63-8d17-4c92729667d0 admin admin] PATH_INFO: `/v3/projects/b4f9a398e4d54c3f9a1822fd2b2c125b/users/6b02d925639a455c844929c6841088fb/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.305715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 907/1826] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:04 2022] GET /identity/v3/roles => generated 1427 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:04.310078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97d260-7f66-4245-98bc-24133dd7b622 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.321119 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947d5a2-c68e-4c63-8d17-4c92729667d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f9a398e4d54c3f9a1822fd2b2c125b, user_id=6b02d925639a455c844929c6841088fb, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.321739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97d260-7f66-4245-98bc-24133dd7b622 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.325177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947d5a2-c68e-4c63-8d17-4c92729667d0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.327525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880344a-2395-493a-bee1-fd84e1dce395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.328220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880344a-2395-493a-bee1-fd84e1dce395 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.328576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880344a-2395-493a-bee1-fd84e1dce395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.329051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880344a-2395-493a-bee1-fd84e1dce395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.329501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880344a-2395-493a-bee1-fd84e1dce395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.331227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880344a-2395-493a-bee1-fd84e1dce395 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.344090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 920/1827] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:04 2022] PUT /identity/v3/projects/b4f9a398e4d54c3f9a1822fd2b2c125b/users/6b02d925639a455c844929c6841088fb/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:04.348814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947d5a2-c68e-4c63-8d17-4c92729667d0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.353190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 908/1828] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:04 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:04.356170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346634-6fac-4759-9c8e-563556edc859 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.356570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346634-6fac-4759-9c8e-563556edc859 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.356857 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346634-6fac-4759-9c8e-563556edc859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.397445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4f346634-6fac-4759-9c8e-563556edc859 None None] MFA Rules not processed for user `6b02d925639a455c844929c6841088fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:04.399384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4f346634-6fac-4759-9c8e-563556edc859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.419332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947d5a2-c68e-4c63-8d17-4c92729667d0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.419545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:04.425169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec7c7b-3588-4e03-b4bb-bc98c4943d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.426003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec7c7b-3588-4e03-b4bb-bc98c4943d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.426381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec7c7b-3588-4e03-b4bb-bc98c4943d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.426861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec7c7b-3588-4e03-b4bb-bc98c4943d38 admin admin] PATH_INFO: `/v3/projects/20845a2305954021a11f4f914ccea2cb/users/f393da2e186a4c419dea5d207ded5f03/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.446881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec7c7b-3588-4e03-b4bb-bc98c4943d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20845a2305954021a11f4f914ccea2cb, user_id=f393da2e186a4c419dea5d207ded5f03, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.450518 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec7c7b-3588-4e03-b4bb-bc98c4943d38 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.473220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 921/1829] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:04 2022] PUT /identity/v3/projects/20845a2305954021a11f4f914ccea2cb/users/f393da2e186a4c419dea5d207ded5f03/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:04.486163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd34c2-55fe-4b03-8efa-f8edebcb7c8e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.486464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd34c2-55fe-4b03-8efa-f8edebcb7c8e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.487767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd34c2-55fe-4b03-8efa-f8edebcb7c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.510358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 909/1830] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:04.512845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f346634-6fac-4759-9c8e-563556edc859 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.526771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6bfd34c2-55fe-4b03-8efa-f8edebcb7c8e None None] MFA Rules not processed for user `f393da2e186a4c419dea5d207ded5f03`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:04.528348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6bfd34c2-55fe-4b03-8efa-f8edebcb7c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.534462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f346634-6fac-4759-9c8e-563556edc859 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.546623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:04.616613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067bdd6-c86b-4815-8b54-ad7610f08b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.617245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067bdd6-c86b-4815-8b54-ad7610f08b07 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.617629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067bdd6-c86b-4815-8b54-ad7610f08b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.617974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067bdd6-c86b-4815-8b54-ad7610f08b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.618670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067bdd6-c86b-4815-8b54-ad7610f08b07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.620951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067bdd6-c86b-4815-8b54-ad7610f08b07 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.627088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 922/1831] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:04 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:04.628004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd34c2-55fe-4b03-8efa-f8edebcb7c8e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.629272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6bfd34c2-55fe-4b03-8efa-f8edebcb7c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.643493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd34c2-55fe-4b03-8efa-f8edebcb7c8e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.649272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.650132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.650950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.651296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] PATH_INFO: `/v3/projects/4cd8d50ce44b40f6a25f7e098ac458be` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.656823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd8d50ce44b40f6a25f7e098ac458be)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.657548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 910/1832] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:04 2022] POST /identity/v3/projects => generated 366 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:04.657674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.657755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067bdd6-c86b-4815-8b54-ad7610f08b07 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.716604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd8d50ce44b40f6a25f7e098ac458be', 'request_id': 'req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.724928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd8d50ce44b40f6a25f7e098ac458be', 'request_id': 'req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.732485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd8d50ce44b40f6a25f7e098ac458be', 'request_id': 'req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.737998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067bdd6-c86b-4815-8b54-ad7610f08b07 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.759458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] The token cache is being invalidate because project 4cd8d50ce44b40f6a25f7e098ac458be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:04.759458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.759458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:04.759458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 923/1833] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:04 2022] DELETE /identity/v3/projects/4cd8d50ce44b40f6a25f7e098ac458be => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:04.759458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:04.761124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.798732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:04.812379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349acda-df00-4b2d-b1e4-dbf01e56518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.813020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349acda-df00-4b2d-b1e4-dbf01e56518d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.813649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349acda-df00-4b2d-b1e4-dbf01e56518d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.813925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349acda-df00-4b2d-b1e4-dbf01e56518d admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.814688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349acda-df00-4b2d-b1e4-dbf01e56518d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.816801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349acda-df00-4b2d-b1e4-dbf01e56518d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:04.869916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 911/1834] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:04 2022] POST /identity/v3/projects => generated 388 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:04.869916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae0299-ab78-4061-b003-aa97c1fa0fe1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.869916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae0299-ab78-4061-b003-aa97c1fa0fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.869916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae0299-ab78-4061-b003-aa97c1fa0fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.872244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c201f-ec1b-433b-8d2d-ec0c7e035294 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:04.915922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5bae0299-ab78-4061-b003-aa97c1fa0fe1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:04.917791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5bae0299-ab78-4061-b003-aa97c1fa0fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:04.929659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:04.975431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee072b1-4c00-481f-beee-c9acdaf1c2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:04.976348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee072b1-4c00-481f-beee-c9acdaf1c2cc admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:04.976807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee072b1-4c00-481f-beee-c9acdaf1c2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:04.977248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee072b1-4c00-481f-beee-c9acdaf1c2cc admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:04.978113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee072b1-4c00-481f-beee-c9acdaf1c2cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:04.980312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee072b1-4c00-481f-beee-c9acdaf1c2cc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.013886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 912/1835] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:05.015051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae0299-ab78-4061-b003-aa97c1fa0fe1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.018178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 924/1836] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:04 2022] POST /identity/v3/users => generated 290 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:05.018956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5bae0299-ab78-4061-b003-aa97c1fa0fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:05.019113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee072b1-4c00-481f-beee-c9acdaf1c2cc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.034298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee072b1-4c00-481f-beee-c9acdaf1c2cc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.042194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a1804-85d3-436c-8562-5a99b6aec339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.042940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a1804-85d3-436c-8562-5a99b6aec339 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.043421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a1804-85d3-436c-8562-5a99b6aec339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.043815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a1804-85d3-436c-8562-5a99b6aec339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.044662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a1804-85d3-436c-8562-5a99b6aec339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.046634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae0299-ab78-4061-b003-aa97c1fa0fe1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.046753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a1804-85d3-436c-8562-5a99b6aec339 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.063775 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04c61d-85a6-4f7d-b09a-17ee3520a7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.064358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 925/1837] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:05 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:05.064440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04c61d-85a6-4f7d-b09a-17ee3520a7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.064678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04c61d-85a6-4f7d-b09a-17ee3520a7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.064954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04c61d-85a6-4f7d-b09a-17ee3520a7ca admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.065515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6daee7-fb39-4e00-a62a-0aaf97e1f6b5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.065634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04c61d-85a6-4f7d-b09a-17ee3520a7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.065839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6daee7-fb39-4e00-a62a-0aaf97e1f6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.066879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6daee7-fb39-4e00-a62a-0aaf97e1f6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.067577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04c61d-85a6-4f7d-b09a-17ee3520a7ca admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.097303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7e6daee7-fb39-4e00-a62a-0aaf97e1f6b5 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:05.098921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7e6daee7-fb39-4e00-a62a-0aaf97e1f6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:05.111423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 926/1838] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:05.112274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6daee7-fb39-4e00-a62a-0aaf97e1f6b5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.122828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 913/1839] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:05 2022] POST /identity/v3/users => generated 301 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:05.123771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04c61d-85a6-4f7d-b09a-17ee3520a7ca admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.133832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6daee7-fb39-4e00-a62a-0aaf97e1f6b5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.144635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0fe09-f808-40db-83a9-cadda4f1a9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.145346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0fe09-f808-40db-83a9-cadda4f1a9a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.145649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04c61d-85a6-4f7d-b09a-17ee3520a7ca admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.147458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0fe09-f808-40db-83a9-cadda4f1a9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.147458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0fe09-f808-40db-83a9-cadda4f1a9a8 admin admin] PATH_INFO: `/v3/projects/604f1f42e4af4119be09ff3dd286d8be/users/25f53d03ba0b4ae6a263d031a750f114/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.158886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987f2de-993c-4783-8275-b0d44f799452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.159699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987f2de-993c-4783-8275-b0d44f799452 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.160055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987f2de-993c-4783-8275-b0d44f799452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.160330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987f2de-993c-4783-8275-b0d44f799452 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.163060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987f2de-993c-4783-8275-b0d44f799452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:05.163537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987f2de-993c-4783-8275-b0d44f799452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.165311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987f2de-993c-4783-8275-b0d44f799452 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.183193 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0fe09-f808-40db-83a9-cadda4f1a9a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=604f1f42e4af4119be09ff3dd286d8be, user_id=25f53d03ba0b4ae6a263d031a750f114, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.186318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0fe09-f808-40db-83a9-cadda4f1a9a8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.201429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 914/1840] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:05.202495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987f2de-993c-4783-8275-b0d44f799452 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.220896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 927/1841] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:05 2022] PUT /identity/v3/projects/604f1f42e4af4119be09ff3dd286d8be/users/25f53d03ba0b4ae6a263d031a750f114/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:05.221866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0fe09-f808-40db-83a9-cadda4f1a9a8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.226135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987f2de-993c-4783-8275-b0d44f799452 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.342222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3659e1-6945-4df3-b557-4ee56e9fc29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.342887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3659e1-6945-4df3-b557-4ee56e9fc29d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.343225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3659e1-6945-4df3-b557-4ee56e9fc29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.343590 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3659e1-6945-4df3-b557-4ee56e9fc29d admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.344311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3659e1-6945-4df3-b557-4ee56e9fc29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.346561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3659e1-6945-4df3-b557-4ee56e9fc29d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.361761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 915/1842] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:05 2022] GET /identity/v3/roles => generated 1427 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:05.365242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3659e1-6945-4df3-b557-4ee56e9fc29d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.378905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3659e1-6945-4df3-b557-4ee56e9fc29d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.404424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0fe09-f808-40db-83a9-cadda4f1a9a8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.408685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b615aa-f161-482a-ab0a-ede9292e03dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.409633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b615aa-f161-482a-ab0a-ede9292e03dd admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.410089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b615aa-f161-482a-ab0a-ede9292e03dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.410584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b615aa-f161-482a-ab0a-ede9292e03dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.411770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b615aa-f161-482a-ab0a-ede9292e03dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.412124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b1d5e-831f-49ad-8453-ef5ce05a83b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.412927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1d5e-831f-49ad-8453-ef5ce05a83b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.413697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1d5e-831f-49ad-8453-ef5ce05a83b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.414391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1d5e-831f-49ad-8453-ef5ce05a83b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.416065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b1d5e-831f-49ad-8453-ef5ce05a83b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.417694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b615aa-f161-482a-ab0a-ede9292e03dd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.420649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b1d5e-831f-49ad-8453-ef5ce05a83b3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.449087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 916/1843] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:05 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:05.450058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b615aa-f161-482a-ab0a-ede9292e03dd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.471163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b615aa-f161-482a-ab0a-ede9292e03dd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.486887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29632cd0-fe46-45e7-bdaa-45278a62cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.487556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 928/1844] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:05 2022] POST /identity/v3/projects => generated 384 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:05.488912 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b1d5e-831f-49ad-8453-ef5ce05a83b3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.494584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29632cd0-fe46-45e7-bdaa-45278a62cd8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.495157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29632cd0-fe46-45e7-bdaa-45278a62cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.495520 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29632cd0-fe46-45e7-bdaa-45278a62cd8c admin admin] PATH_INFO: `/v3/projects/816921fb389346bfad7895f69e6af00e/users/a15070e619f44b968240a76699e250fa/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.515971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29632cd0-fe46-45e7-bdaa-45278a62cd8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=816921fb389346bfad7895f69e6af00e, user_id=a15070e619f44b968240a76699e250fa, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.519359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29632cd0-fe46-45e7-bdaa-45278a62cd8c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.546403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 917/1845] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:05 2022] PUT /identity/v3/projects/816921fb389346bfad7895f69e6af00e/users/a15070e619f44b968240a76699e250fa/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:05.547395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29632cd0-fe46-45e7-bdaa-45278a62cd8c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.558096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:05.605879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b1d5e-831f-49ad-8453-ef5ce05a83b3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.640142 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29632cd0-fe46-45e7-bdaa-45278a62cd8c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.644486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:05.650153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a2f88-29f1-4050-826c-52448c657195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.651908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a2f88-29f1-4050-826c-52448c657195 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.652227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a2f88-29f1-4050-826c-52448c657195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.652802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a2f88-29f1-4050-826c-52448c657195 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.653709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a2f88-29f1-4050-826c-52448c657195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.655710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a2f88-29f1-4050-826c-52448c657195 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.688822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 918/1846] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:05 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:05.688822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a2f88-29f1-4050-826c-52448c657195 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.698909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a2f88-29f1-4050-826c-52448c657195 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.705697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd41a3-5c85-4380-be52-0270de828e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.706340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd41a3-5c85-4380-be52-0270de828e95 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.706624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd41a3-5c85-4380-be52-0270de828e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.706880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd41a3-5c85-4380-be52-0270de828e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.707535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd41a3-5c85-4380-be52-0270de828e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.709182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd41a3-5c85-4380-be52-0270de828e95 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.722686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 919/1847] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:05 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:05.723683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd41a3-5c85-4380-be52-0270de828e95 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.724056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046bfd2-b804-44b5-ae0d-8a1ccd60d2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.724956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046bfd2-b804-44b5-ae0d-8a1ccd60d2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.725498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046bfd2-b804-44b5-ae0d-8a1ccd60d2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.725971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046bfd2-b804-44b5-ae0d-8a1ccd60d2e4 admin admin] PATH_INFO: `/v3/projects/604f1f42e4af4119be09ff3dd286d8be/users/25f53d03ba0b4ae6a263d031a750f114/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.731792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046bfd2-b804-44b5-ae0d-8a1ccd60d2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=604f1f42e4af4119be09ff3dd286d8be, user_id=25f53d03ba0b4ae6a263d031a750f114, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.735962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046bfd2-b804-44b5-ae0d-8a1ccd60d2e4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.756612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd41a3-5c85-4380-be52-0270de828e95 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.769749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 929/1848] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:05 2022] PUT /identity/v3/projects/604f1f42e4af4119be09ff3dd286d8be/users/25f53d03ba0b4ae6a263d031a750f114/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:05.770776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046bfd2-b804-44b5-ae0d-8a1ccd60d2e4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.881256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e79ffb-a631-46c2-ad89-ad10391148ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.881954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e79ffb-a631-46c2-ad89-ad10391148ba admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.882502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e79ffb-a631-46c2-ad89-ad10391148ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.882782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e79ffb-a631-46c2-ad89-ad10391148ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.883429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e79ffb-a631-46c2-ad89-ad10391148ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.885061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e79ffb-a631-46c2-ad89-ad10391148ba admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.889628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046bfd2-b804-44b5-ae0d-8a1ccd60d2e4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:05.895045 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe259e30-006e-45b7-8c81-5d6c6128fc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:05.895966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe259e30-006e-45b7-8c81-5d6c6128fc8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.896253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe259e30-006e-45b7-8c81-5d6c6128fc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.896529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe259e30-006e-45b7-8c81-5d6c6128fc8a admin admin] PATH_INFO: `/v3/projects/816921fb389346bfad7895f69e6af00e/users/a15070e619f44b968240a76699e250fa/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.896688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 920/1849] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:05 2022] GET /identity/v3/roles => generated 1427 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:05.898542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe259e30-006e-45b7-8c81-5d6c6128fc8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=816921fb389346bfad7895f69e6af00e, user_id=a15070e619f44b968240a76699e250fa, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:05.899210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e98f74-e62d-4740-ade1-c087e0df46e2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:05.899725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e98f74-e62d-4740-ade1-c087e0df46e2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:05.900227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e98f74-e62d-4740-ade1-c087e0df46e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:05.901412 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe259e30-006e-45b7-8c81-5d6c6128fc8a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:05.913543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 930/1850] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:05 2022] PUT /identity/v3/projects/816921fb389346bfad7895f69e6af00e/users/a15070e619f44b968240a76699e250fa/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:05.914416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe259e30-006e-45b7-8c81-5d6c6128fc8a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:05.955045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d5e98f74-e62d-4740-ade1-c087e0df46e2 None None] MFA Rules not processed for user `25f53d03ba0b4ae6a263d031a750f114`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:05.956585 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d5e98f74-e62d-4740-ade1-c087e0df46e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:06.008325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe259e30-006e-45b7-8c81-5d6c6128fc8a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.014774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3c9b8-d28e-4042-a127-596e69a65053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.015481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3c9b8-d28e-4042-a127-596e69a65053 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.015882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3c9b8-d28e-4042-a127-596e69a65053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.016259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3c9b8-d28e-4042-a127-596e69a65053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.017040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3c9b8-d28e-4042-a127-596e69a65053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.019271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3c9b8-d28e-4042-a127-596e69a65053 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.033000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 931/1851] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:05 2022] GET /identity/v3/roles => generated 1427 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:06.034354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f981-36a4-4bb3-a1d8-5912c003c039 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.034756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f981-36a4-4bb3-a1d8-5912c003c039 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.035127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f981-36a4-4bb3-a1d8-5912c003c039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.048174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 921/1852] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:06.049100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e98f74-e62d-4740-ade1-c087e0df46e2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.055155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:06.063208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e98f74-e62d-4740-ade1-c087e0df46e2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.068684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-67b3f981-36a4-4bb3-a1d8-5912c003c039 None None] MFA Rules not processed for user `a15070e619f44b968240a76699e250fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:06.069774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a779dd5-a431-4178-9aa9-6bde7a511a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.070290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-67b3f981-36a4-4bb3-a1d8-5912c003c039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:06.070638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a779dd5-a431-4178-9aa9-6bde7a511a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.071004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a779dd5-a431-4178-9aa9-6bde7a511a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.071388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a779dd5-a431-4178-9aa9-6bde7a511a71 admin admin] PATH_INFO: `/v3/projects/82d0a188969441129bd72a3477f2f8bd/users/8f4832ff60ba4d3283913f2b0c157ec2/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.089565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a779dd5-a431-4178-9aa9-6bde7a511a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d0a188969441129bd72a3477f2f8bd, user_id=8f4832ff60ba4d3283913f2b0c157ec2, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.092438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a779dd5-a431-4178-9aa9-6bde7a511a71 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.107398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 922/1853] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:06 2022] PUT /identity/v3/projects/82d0a188969441129bd72a3477f2f8bd/users/8f4832ff60ba4d3283913f2b0c157ec2/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:06.107933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a779dd5-a431-4178-9aa9-6bde7a511a71 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.109468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2a779dd5-a431-4178-9aa9-6bde7a511a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:06.112541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:06.144962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 932/1854] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:06 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:06.146665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2b4c-6a69-4ab1-b5f2-82e7b001667d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.146968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2b4c-6a69-4ab1-b5f2-82e7b001667d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.147289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2b4c-6a69-4ab1-b5f2-82e7b001667d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.176593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-93ef2b4c-6a69-4ab1-b5f2-82e7b001667d None None] MFA Rules not processed for user `8f4832ff60ba4d3283913f2b0c157ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:06.177839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-93ef2b4c-6a69-4ab1-b5f2-82e7b001667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:06.179745 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a779dd5-a431-4178-9aa9-6bde7a511a71 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.183464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2aee62-92f9-4fb2-a607-0d0e23966ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.184076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2aee62-92f9-4fb2-a607-0d0e23966ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.184359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2aee62-92f9-4fb2-a607-0d0e23966ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.184633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2aee62-92f9-4fb2-a607-0d0e23966ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.251389 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 933/1855] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:06.256121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2aee62-92f9-4fb2-a607-0d0e23966ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.256327 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2b4c-6a69-4ab1-b5f2-82e7b001667d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.257882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2aee62-92f9-4fb2-a607-0d0e23966ff1 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.258063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-93ef2b4c-6a69-4ab1-b5f2-82e7b001667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:06.269682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 923/1856] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:06.270496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2aee62-92f9-4fb2-a607-0d0e23966ff1 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.270913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2b4c-6a69-4ab1-b5f2-82e7b001667d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.275096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667386d-e022-44ad-a111-009cc44281e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.276358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667386d-e022-44ad-a111-009cc44281e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.276748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667386d-e022-44ad-a111-009cc44281e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.277124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667386d-e022-44ad-a111-009cc44281e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.367990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667386d-e022-44ad-a111-009cc44281e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.370920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667386d-e022-44ad-a111-009cc44281e0 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.373056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2aee62-92f9-4fb2-a607-0d0e23966ff1 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.379301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003dbee-0c73-4bc6-8448-4d381d4343c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.379838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003dbee-0c73-4bc6-8448-4d381d4343c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.380153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003dbee-0c73-4bc6-8448-4d381d4343c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.380426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003dbee-0c73-4bc6-8448-4d381d4343c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.381061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003dbee-0c73-4bc6-8448-4d381d4343c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.383151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003dbee-0c73-4bc6-8448-4d381d4343c2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.384106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 934/1857] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:06 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:06.384932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667386d-e022-44ad-a111-009cc44281e0 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.393611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667386d-e022-44ad-a111-009cc44281e0 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.397435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099cec3-ef73-4bc7-afa9-0832ba2c9b25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.398065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099cec3-ef73-4bc7-afa9-0832ba2c9b25 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.398343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099cec3-ef73-4bc7-afa9-0832ba2c9b25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.398614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099cec3-ef73-4bc7-afa9-0832ba2c9b25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.401415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c099cec3-ef73-4bc7-afa9-0832ba2c9b25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:06.420584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 924/1858] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:06 2022] POST /identity/v3/projects => generated 384 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:06.422566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003dbee-0c73-4bc6-8448-4d381d4343c2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.443215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:06.485526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:06.490396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099cec3-ef73-4bc7-afa9-0832ba2c9b25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.493738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099cec3-ef73-4bc7-afa9-0832ba2c9b25 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.508240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003dbee-0c73-4bc6-8448-4d381d4343c2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.516171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6beef-f9ed-41c8-95f1-d3af5d6007ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.517004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6beef-f9ed-41c8-95f1-d3af5d6007ce admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.517418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6beef-f9ed-41c8-95f1-d3af5d6007ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.517917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6beef-f9ed-41c8-95f1-d3af5d6007ce admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.519091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6beef-f9ed-41c8-95f1-d3af5d6007ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.521011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6beef-f9ed-41c8-95f1-d3af5d6007ce admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.522629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 935/1859] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:06.548830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 925/1860] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:06 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:06.553670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099cec3-ef73-4bc7-afa9-0832ba2c9b25 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.566902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099cec3-ef73-4bc7-afa9-0832ba2c9b25 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.572650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63577065-bca5-418e-8bf9-28dfaaedd98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.573371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63577065-bca5-418e-8bf9-28dfaaedd98a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.573751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63577065-bca5-418e-8bf9-28dfaaedd98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.574036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63577065-bca5-418e-8bf9-28dfaaedd98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.574736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63577065-bca5-418e-8bf9-28dfaaedd98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.576647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63577065-bca5-418e-8bf9-28dfaaedd98a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.589627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 936/1861] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:06 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:06.593696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6beef-f9ed-41c8-95f1-d3af5d6007ce admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.607527 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6beef-f9ed-41c8-95f1-d3af5d6007ce admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.612811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4302a8d-289a-4da2-ba7b-7f131be39c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.618629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4302a8d-289a-4da2-ba7b-7f131be39c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.619208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4302a8d-289a-4da2-ba7b-7f131be39c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.619340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4302a8d-289a-4da2-ba7b-7f131be39c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.620520 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4302a8d-289a-4da2-ba7b-7f131be39c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.622186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4302a8d-289a-4da2-ba7b-7f131be39c96 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.637997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 926/1862] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:06 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:06.639693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63577065-bca5-418e-8bf9-28dfaaedd98a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.655681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63577065-bca5-418e-8bf9-28dfaaedd98a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.676688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6eabcc-5ef2-4a9e-8c18-172a3ef928aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.676976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6eabcc-5ef2-4a9e-8c18-172a3ef928aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.677255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6eabcc-5ef2-4a9e-8c18-172a3ef928aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.677528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6eabcc-5ef2-4a9e-8c18-172a3ef928aa admin admin] PATH_INFO: `/v3/projects/150f5a45d1624162b377a4a34a3bad9b/users/301303a6633c4e71ad4e2011495c4392/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.707611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6eabcc-5ef2-4a9e-8c18-172a3ef928aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=150f5a45d1624162b377a4a34a3bad9b, user_id=301303a6633c4e71ad4e2011495c4392, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.714208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6eabcc-5ef2-4a9e-8c18-172a3ef928aa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:06.733997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 937/1863] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:06 2022] PUT /identity/v3/projects/150f5a45d1624162b377a4a34a3bad9b/users/301303a6633c4e71ad4e2011495c4392/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:06.753061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a0600-d13d-4b70-9982-983fd906e375 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.753599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a0600-d13d-4b70-9982-983fd906e375 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.754025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a0600-d13d-4b70-9982-983fd906e375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.788854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e39a0600-d13d-4b70-9982-983fd906e375 None None] MFA Rules not processed for user `301303a6633c4e71ad4e2011495c4392`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:06.790641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e39a0600-d13d-4b70-9982-983fd906e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:06.891494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 927/1864] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:06.896663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6eabcc-5ef2-4a9e-8c18-172a3ef928aa admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:06.978618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6eabcc-5ef2-4a9e-8c18-172a3ef928aa admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:06.986568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be0a6e-a815-4d98-845c-9685f4f37f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:06.987197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be0a6e-a815-4d98-845c-9685f4f37f3f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:06.987482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be0a6e-a815-4d98-845c-9685f4f37f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:06.987755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be0a6e-a815-4d98-845c-9685f4f37f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:06.988445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be0a6e-a815-4d98-845c-9685f4f37f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:06.990412 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be0a6e-a815-4d98-845c-9685f4f37f3f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:07.029062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 938/1865] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:06 2022] POST /identity/v3/projects => generated 386 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:07.032726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a0600-d13d-4b70-9982-983fd906e375 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:07.048210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a0600-d13d-4b70-9982-983fd906e375 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:07.128921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be0a6e-a815-4d98-845c-9685f4f37f3f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:07.152931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718446f1-1228-4d02-8898-5590ee69eb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:07.153674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718446f1-1228-4d02-8898-5590ee69eb4d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:07.154495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718446f1-1228-4d02-8898-5590ee69eb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:07.154495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718446f1-1228-4d02-8898-5590ee69eb4d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:07.155471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718446f1-1228-4d02-8898-5590ee69eb4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:07.159878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718446f1-1228-4d02-8898-5590ee69eb4d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:07.194976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 928/1866] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:07 2022] POST /identity/v3/users => generated 300 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:07.200243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718446f1-1228-4d02-8898-5590ee69eb4d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:07.213753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be0a6e-a815-4d98-845c-9685f4f37f3f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:07.217838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718446f1-1228-4d02-8898-5590ee69eb4d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:07.218372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acedc89-3f79-4c81-bec5-20cb154bbb6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:07.219176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acedc89-3f79-4c81-bec5-20cb154bbb6b service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:07.219546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acedc89-3f79-4c81-bec5-20cb154bbb6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:07.219705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acedc89-3f79-4c81-bec5-20cb154bbb6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:07.221712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2acedc89-3f79-4c81-bec5-20cb154bbb6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:07.223561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed75ae-487d-45e5-bb1b-38a658d0c19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:07.224311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed75ae-487d-45e5-bb1b-38a658d0c19f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:07.224477 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed75ae-487d-45e5-bb1b-38a658d0c19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:07.224689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed75ae-487d-45e5-bb1b-38a658d0c19f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:07.225360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed75ae-487d-45e5-bb1b-38a658d0c19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:07.227100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed75ae-487d-45e5-bb1b-38a658d0c19f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:07.234058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:07.241655 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 929/1867] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:07 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:07.245725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed75ae-487d-45e5-bb1b-38a658d0c19f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:07.256976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed75ae-487d-45e5-bb1b-38a658d0c19f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:07.267627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa176c29-132a-455f-a112-c5b2a1b38ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:07.268311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa176c29-132a-455f-a112-c5b2a1b38ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:07.268594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa176c29-132a-455f-a112-c5b2a1b38ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:07.268873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa176c29-132a-455f-a112-c5b2a1b38ec4 admin admin] PATH_INFO: `/v3/projects/dc92bc3246394aec9dfe9d4673c77dfb/users/5fb089ba65f64e0ba3df9e84f0179e70/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:07.285882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa176c29-132a-455f-a112-c5b2a1b38ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc92bc3246394aec9dfe9d4673c77dfb, user_id=5fb089ba65f64e0ba3df9e84f0179e70, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:07.288850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa176c29-132a-455f-a112-c5b2a1b38ec4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:07.296761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acedc89-3f79-4c81-bec5-20cb154bbb6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:07.298246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acedc89-3f79-4c81-bec5-20cb154bbb6b service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:07.303106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:07.307395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 930/1868] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:07 2022] PUT /identity/v3/projects/dc92bc3246394aec9dfe9d4673c77dfb/users/5fb089ba65f64e0ba3df9e84f0179e70/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:07.310202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 939/1869] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:07.311647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa176c29-132a-455f-a112-c5b2a1b38ec4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:07.378983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa176c29-132a-455f-a112-c5b2a1b38ec4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:07.383748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096d0d1-79f6-46b7-bf81-e125a739fb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:07.384364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096d0d1-79f6-46b7-bf81-e125a739fb2f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:07.385189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096d0d1-79f6-46b7-bf81-e125a739fb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:07.385472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096d0d1-79f6-46b7-bf81-e125a739fb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:07.386174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096d0d1-79f6-46b7-bf81-e125a739fb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:07.387810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096d0d1-79f6-46b7-bf81-e125a739fb2f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:07.404798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 931/1870] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:07 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:07.405879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acedc89-3f79-4c81-bec5-20cb154bbb6b service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:07.417564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acedc89-3f79-4c81-bec5-20cb154bbb6b service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:07.422852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eff01-7d01-43e7-928b-24efd7887a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:07.423559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eff01-7d01-43e7-928b-24efd7887a58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:07.423788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eff01-7d01-43e7-928b-24efd7887a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:07.424142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eff01-7d01-43e7-928b-24efd7887a58 admin admin] PATH_INFO: `/v3/projects/dc92bc3246394aec9dfe9d4673c77dfb/users/5fb089ba65f64e0ba3df9e84f0179e70/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:07.426121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456eff01-7d01-43e7-928b-24efd7887a58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc92bc3246394aec9dfe9d4673c77dfb, user_id=5fb089ba65f64e0ba3df9e84f0179e70, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:07.428803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456eff01-7d01-43e7-928b-24efd7887a58 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:07.442494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 940/1871] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:07 2022] PUT /identity/v3/projects/dc92bc3246394aec9dfe9d4673c77dfb/users/5fb089ba65f64e0ba3df9e84f0179e70/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:07.454127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec963eb-0697-47f7-ace2-93620001c1ba None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:07.454310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec963eb-0697-47f7-ace2-93620001c1ba None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:07.454698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec963eb-0697-47f7-ace2-93620001c1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:07.487065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2ec963eb-0697-47f7-ace2-93620001c1ba None None] MFA Rules not processed for user `5fb089ba65f64e0ba3df9e84f0179e70`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:07.488382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2ec963eb-0697-47f7-ace2-93620001c1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:07.591189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 932/1872] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:07 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:07.609117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eff01-7d01-43e7-928b-24efd7887a58 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:07.694684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eff01-7d01-43e7-928b-24efd7887a58 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:07.701120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba58d5-c4ba-4249-8198-8c747bae32d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:07.701785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba58d5-c4ba-4249-8198-8c747bae32d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:07.702068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba58d5-c4ba-4249-8198-8c747bae32d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:07.702405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba58d5-c4ba-4249-8198-8c747bae32d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:07.779170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba58d5-c4ba-4249-8198-8c747bae32d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:07.780701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba58d5-c4ba-4249-8198-8c747bae32d5 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:07.791665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 941/1873] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:11.841380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec963eb-0697-47f7-ace2-93620001c1ba None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:11.852621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec963eb-0697-47f7-ace2-93620001c1ba None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:11.857047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257a6a9-b283-4361-bcce-c9a8e0fbd181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:11.857704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257a6a9-b283-4361-bcce-c9a8e0fbd181 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:11.857989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257a6a9-b283-4361-bcce-c9a8e0fbd181 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:11.858263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257a6a9-b283-4361-bcce-c9a8e0fbd181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:11.860201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8257a6a9-b283-4361-bcce-c9a8e0fbd181 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:11.935628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:11.955374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257a6a9-b283-4361-bcce-c9a8e0fbd181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:11.957741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257a6a9-b283-4361-bcce-c9a8e0fbd181 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:11.969725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 933/1874] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:12.544615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba58d5-c4ba-4249-8198-8c747bae32d5 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:12.553413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba58d5-c4ba-4249-8198-8c747bae32d5 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:12.557945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d13a3a-87c7-42f0-ae83-f7c1f7c0a2e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:12.558552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d13a3a-87c7-42f0-ae83-f7c1f7c0a2e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:12.558828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d13a3a-87c7-42f0-ae83-f7c1f7c0a2e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:12.559125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d13a3a-87c7-42f0-ae83-f7c1f7c0a2e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:12.631686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d13a3a-87c7-42f0-ae83-f7c1f7c0a2e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:12.633493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d13a3a-87c7-42f0-ae83-f7c1f7c0a2e2 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:12.646611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 942/1875] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:13.925274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257a6a9-b283-4361-bcce-c9a8e0fbd181 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:13.934834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257a6a9-b283-4361-bcce-c9a8e0fbd181 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:13.938511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39853b-64ac-4de9-829f-77669fa6a4ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:13.939109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39853b-64ac-4de9-829f-77669fa6a4ff service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:13.939386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39853b-64ac-4de9-829f-77669fa6a4ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:13.939654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39853b-64ac-4de9-829f-77669fa6a4ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:13.957126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39853b-64ac-4de9-829f-77669fa6a4ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:13.958593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39853b-64ac-4de9-829f-77669fa6a4ff service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:13.969038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 934/1876] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:13 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:17.153033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d13a3a-87c7-42f0-ae83-f7c1f7c0a2e2 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:17.162006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d13a3a-87c7-42f0-ae83-f7c1f7c0a2e2 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:17.165808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a0001-bd16-4446-861e-6eec38870d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:17.166411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0001-bd16-4446-861e-6eec38870d59 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:17.166690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0001-bd16-4446-861e-6eec38870d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:17.166959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0001-bd16-4446-861e-6eec38870d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:17.168666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a87a0001-bd16-4446-861e-6eec38870d59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:17.210525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:17.234807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a0001-bd16-4446-861e-6eec38870d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:17.236381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a0001-bd16-4446-861e-6eec38870d59 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:17.245762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 943/1877] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:17 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:20.202995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d112b-428e-46da-bb3a-3c6aafcd6dad None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:20.204030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d112b-428e-46da-bb3a-3c6aafcd6dad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:20.204600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d112b-428e-46da-bb3a-3c6aafcd6dad None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:20.206464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 935/1878] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:37:20 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:37:20.222392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bad180-524a-49d6-bd35-aebc5802ec40 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:20.222956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bad180-524a-49d6-bd35-aebc5802ec40 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:20.222956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bad180-524a-49d6-bd35-aebc5802ec40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:20.273382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c6bad180-524a-49d6-bd35-aebc5802ec40 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:20.278983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c6bad180-524a-49d6-bd35-aebc5802ec40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:20.515162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 944/1879] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:20.765286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d112b-428e-46da-bb3a-3c6aafcd6dad None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:20.765286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-023d112b-428e-46da-bb3a-3c6aafcd6dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:20.791218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d112b-428e-46da-bb3a-3c6aafcd6dad None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:20.802812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f75d7-4179-4979-9d3b-98f3f718cc8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:20.804094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f75d7-4179-4979-9d3b-98f3f718cc8c service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:20.804447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f75d7-4179-4979-9d3b-98f3f718cc8c service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:20.804807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f75d7-4179-4979-9d3b-98f3f718cc8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:20.829882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f75d7-4179-4979-9d3b-98f3f718cc8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:20.829882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f75d7-4179-4979-9d3b-98f3f718cc8c service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:20.856838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 936/1880] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:20 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:21.005858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bad180-524a-49d6-bd35-aebc5802ec40 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:21.097306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bad180-524a-49d6-bd35-aebc5802ec40 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:21.103431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:21.104336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:21.104679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:21.104958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] PATH_INFO: `/v3/users/d5bd874497264bba806d407b655cf7e4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:21.106172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5bd874497264bba806d407b655cf7e4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:21.108510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:21.172216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5bd874497264bba806d407b655cf7e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.190393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5bd874497264bba806d407b655cf7e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.195870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:21.199029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5bd874497264bba806d407b655cf7e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.207505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5bd874497264bba806d407b655cf7e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.208200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5bd874497264bba806d407b655cf7e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.214586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 945/1881] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:21 2022] DELETE /identity/v3/users/d5bd874497264bba806d407b655cf7e4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:21.219093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06c823-ec24-4cb6-986f-0c6ef81c7a91 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:21.219654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06c823-ec24-4cb6-986f-0c6ef81c7a91 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:21.219654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06c823-ec24-4cb6-986f-0c6ef81c7a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:21.242294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2d06c823-ec24-4cb6-986f-0c6ef81c7a91 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:21.244096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2d06c823-ec24-4cb6-986f-0c6ef81c7a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:21.337490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 937/1882] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:21.349039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:21.352207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:21.424246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30577100-8ebc-48f8-891f-468ba59cc13a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:21.428255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc0e05-239f-4309-88a0-7136fc5d4694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:21.433880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc0e05-239f-4309-88a0-7136fc5d4694 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:21.433880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc0e05-239f-4309-88a0-7136fc5d4694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:21.433880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc0e05-239f-4309-88a0-7136fc5d4694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:21.442211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc0e05-239f-4309-88a0-7136fc5d4694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:21.443647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc0e05-239f-4309-88a0-7136fc5d4694 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:21.453746 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 946/1883] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:21.742424 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06c823-ec24-4cb6-986f-0c6ef81c7a91 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:21.743938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2d06c823-ec24-4cb6-986f-0c6ef81c7a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:21.756821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06c823-ec24-4cb6-986f-0c6ef81c7a91 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:21.762199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:21.762837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:21.763108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:21.763381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] PATH_INFO: `/v3/projects/346083745ed84ee198df340431a6bc7a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:21.764496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=346083745ed84ee198df340431a6bc7a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:21.766416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:21.820662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '346083745ed84ee198df340431a6bc7a', 'request_id': 'req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.827754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '346083745ed84ee198df340431a6bc7a', 'request_id': 'req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.833512 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '346083745ed84ee198df340431a6bc7a', 'request_id': 'req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.849057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] The token cache is being invalidate because project 346083745ed84ee198df340431a6bc7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:21.849457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.850388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:21.852100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 938/1884] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:21 2022] DELETE /identity/v3/projects/346083745ed84ee198df340431a6bc7a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:21.856388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc0e05-239f-4309-88a0-7136fc5d4694 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:21.866618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc0e05-239f-4309-88a0-7136fc5d4694 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:21.935730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:21.936652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:21.936930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:21.937199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] PATH_INFO: `/v3/users/f393da2e186a4c419dea5d207ded5f03` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:21.938403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f393da2e186a4c419dea5d207ded5f03)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:21.945338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:21.989927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f393da2e186a4c419dea5d207ded5f03'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.008934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f393da2e186a4c419dea5d207ded5f03'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.016965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f393da2e186a4c419dea5d207ded5f03'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.026636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f393da2e186a4c419dea5d207ded5f03'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.027274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f393da2e186a4c419dea5d207ded5f03'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.033381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 947/1885] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:21 2022] DELETE /identity/v3/users/f393da2e186a4c419dea5d207ded5f03 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:22.388502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:22.390707 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:22.414260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f06078-53d0-4228-ba21-e8ac7c16fb6d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:22.473568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:22.487784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:22.488422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:22.488723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:22.489000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] PATH_INFO: `/v3/projects/20845a2305954021a11f4f914ccea2cb` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:22.490145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] RBAC: Authorizing `identity:delete_project(project_id=20845a2305954021a11f4f914ccea2cb)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:22.492044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:22.539514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20845a2305954021a11f4f914ccea2cb', 'request_id': 'req-4089e98e-493c-4c81-b1b3-6801725d01be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.545832 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20845a2305954021a11f4f914ccea2cb', 'request_id': 'req-4089e98e-493c-4c81-b1b3-6801725d01be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.551443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20845a2305954021a11f4f914ccea2cb', 'request_id': 'req-4089e98e-493c-4c81-b1b3-6801725d01be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.564990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] The token cache is being invalidate because project 20845a2305954021a11f4f914ccea2cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:22.565279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.565910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.567522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 939/1886] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:22 2022] DELETE /identity/v3/projects/20845a2305954021a11f4f914ccea2cb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:22.572333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:22.574229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:22.587531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be946ad-461c-4bcd-bc1a-a746cdd77505 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:22.595590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:22.660897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:22.661858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:22.662141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:22.662409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] PATH_INFO: `/v3/users/a15070e619f44b968240a76699e250fa` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:22.663569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15070e619f44b968240a76699e250fa)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:22.665728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:22.714303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15070e619f44b968240a76699e250fa'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.726594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15070e619f44b968240a76699e250fa'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.733634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15070e619f44b968240a76699e250fa'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.742037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15070e619f44b968240a76699e250fa'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.742683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-673877b5-a717-4b48-b099-f729791c9a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15070e619f44b968240a76699e250fa'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:22.749398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 948/1887] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:22 2022] DELETE /identity/v3/users/a15070e619f44b968240a76699e250fa => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:23.027068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:23.030284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:23.067100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089e98e-493c-4c81-b1b3-6801725d01be admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:23.146529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:23.147167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.147443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.147715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] PATH_INFO: `/v3/projects/816921fb389346bfad7895f69e6af00e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.149131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=816921fb389346bfad7895f69e6af00e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:23.151955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:23.204419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '816921fb389346bfad7895f69e6af00e', 'request_id': 'req-7f587148-525b-4b4f-905b-e495f5a963bf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:23.211034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '816921fb389346bfad7895f69e6af00e', 'request_id': 'req-7f587148-525b-4b4f-905b-e495f5a963bf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:23.218279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '816921fb389346bfad7895f69e6af00e', 'request_id': 'req-7f587148-525b-4b4f-905b-e495f5a963bf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:23.231531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] The token cache is being invalidate because project 816921fb389346bfad7895f69e6af00e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:23.232733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:23.232733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f587148-525b-4b4f-905b-e495f5a963bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:23.234284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 940/1888] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:23 2022] DELETE /identity/v3/projects/816921fb389346bfad7895f69e6af00e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:23.240014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23b5c0-0671-4b3c-abad-0cd7f77695a1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.240303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23b5c0-0671-4b3c-abad-0cd7f77695a1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.240584 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23b5c0-0671-4b3c-abad-0cd7f77695a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.260449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4f23b5c0-0671-4b3c-abad-0cd7f77695a1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:23.261702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4f23b5c0-0671-4b3c-abad-0cd7f77695a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:23.337328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 949/1889] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:23.346145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96b8d3-2fe8-4d68-91bb-411caab6dc33 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.346429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96b8d3-2fe8-4d68-91bb-411caab6dc33 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.346697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96b8d3-2fe8-4d68-91bb-411caab6dc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.366992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fa96b8d3-2fe8-4d68-91bb-411caab6dc33 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:23.368420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fa96b8d3-2fe8-4d68-91bb-411caab6dc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:23.378569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 941/1890] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:23.390110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23b5c0-0671-4b3c-abad-0cd7f77695a1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:23.410313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23b5c0-0671-4b3c-abad-0cd7f77695a1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:23.418060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6070aa-3b56-4780-9cba-eeb15822f8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:23.418710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6070aa-3b56-4780-9cba-eeb15822f8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.418985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6070aa-3b56-4780-9cba-eeb15822f8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.419259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6070aa-3b56-4780-9cba-eeb15822f8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.419934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6070aa-3b56-4780-9cba-eeb15822f8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:23.420241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6070aa-3b56-4780-9cba-eeb15822f8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:23.422115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6070aa-3b56-4780-9cba-eeb15822f8d1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:23.437875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 950/1891] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:23.442399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96b8d3-2fe8-4d68-91bb-411caab6dc33 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:23.456213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:23.463938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96b8d3-2fe8-4d68-91bb-411caab6dc33 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:23.479257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6a265-7eb2-4223-a517-6b3a9e6dc667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:23.479257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6a265-7eb2-4223-a517-6b3a9e6dc667 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.479610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6a265-7eb2-4223-a517-6b3a9e6dc667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.479819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6a265-7eb2-4223-a517-6b3a9e6dc667 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.480480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6a265-7eb2-4223-a517-6b3a9e6dc667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:23.482662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6a265-7eb2-4223-a517-6b3a9e6dc667 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:23.562630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 942/1892] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:23 2022] POST /identity/v3/projects => generated 390 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:23.567404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6070aa-3b56-4780-9cba-eeb15822f8d1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:23.579551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6070aa-3b56-4780-9cba-eeb15822f8d1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:23.601927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:23.687004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2682401-b8e5-419f-a325-a99bff2a3281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:23.689171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2682401-b8e5-419f-a325-a99bff2a3281 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.689760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2682401-b8e5-419f-a325-a99bff2a3281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.690376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2682401-b8e5-419f-a325-a99bff2a3281 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.691492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2682401-b8e5-419f-a325-a99bff2a3281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:23.694366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2682401-b8e5-419f-a325-a99bff2a3281 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:23.742627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 951/1893] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:23 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:23.747323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6a265-7eb2-4223-a517-6b3a9e6dc667 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:23.764245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6a265-7eb2-4223-a517-6b3a9e6dc667 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:23.783720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728429dd-3ab3-43ad-94e5-f7d24197b049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:23.784538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728429dd-3ab3-43ad-94e5-f7d24197b049 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.785019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728429dd-3ab3-43ad-94e5-f7d24197b049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.785310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728429dd-3ab3-43ad-94e5-f7d24197b049 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.786109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728429dd-3ab3-43ad-94e5-f7d24197b049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:23.791312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728429dd-3ab3-43ad-94e5-f7d24197b049 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:23.819345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 943/1894] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:23 2022] GET /identity/v3/roles => generated 1427 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:23.820583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2682401-b8e5-419f-a325-a99bff2a3281 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:23.833613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2682401-b8e5-419f-a325-a99bff2a3281 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:23.841607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced673d-0e59-4879-9158-62a168d7b0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:23.842391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced673d-0e59-4879-9158-62a168d7b0ea admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.842681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced673d-0e59-4879-9158-62a168d7b0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.842974 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced673d-0e59-4879-9158-62a168d7b0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.843809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced673d-0e59-4879-9158-62a168d7b0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:23.845771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced673d-0e59-4879-9158-62a168d7b0ea admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:23.859457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 952/1895] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:23 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:23.864251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728429dd-3ab3-43ad-94e5-f7d24197b049 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:23.885234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728429dd-3ab3-43ad-94e5-f7d24197b049 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:23.894002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee804400-f53b-4c8d-b601-d29d6792286a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:23.895027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee804400-f53b-4c8d-b601-d29d6792286a admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.895456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee804400-f53b-4c8d-b601-d29d6792286a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.895878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee804400-f53b-4c8d-b601-d29d6792286a admin admin] PATH_INFO: `/v3/projects/a93b6a1166cd4da9b6bd53cc5731c851/users/1825ef17f0e14e2e8015043ad2316d82/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.919906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee804400-f53b-4c8d-b601-d29d6792286a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93b6a1166cd4da9b6bd53cc5731c851, user_id=1825ef17f0e14e2e8015043ad2316d82, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:23.922747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee804400-f53b-4c8d-b601-d29d6792286a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:23.943371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 944/1896] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:23 2022] PUT /identity/v3/projects/a93b6a1166cd4da9b6bd53cc5731c851/users/1825ef17f0e14e2e8015043ad2316d82/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:23.955409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8c568-7802-4160-b84d-b9726e5571bd None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:23.955756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8c568-7802-4160-b84d-b9726e5571bd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:23.955999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8c568-7802-4160-b84d-b9726e5571bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:23.988241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3df8c568-7802-4160-b84d-b9726e5571bd None None] MFA Rules not processed for user `1825ef17f0e14e2e8015043ad2316d82`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:23.989930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3df8c568-7802-4160-b84d-b9726e5571bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:24.084010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 953/1897] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:24.088896 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee804400-f53b-4c8d-b601-d29d6792286a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:24.191448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee804400-f53b-4c8d-b601-d29d6792286a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:24.196372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968df502-8141-41f5-b492-9ecefaa5450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:24.197835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968df502-8141-41f5-b492-9ecefaa5450b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:24.198311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968df502-8141-41f5-b492-9ecefaa5450b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:24.198588 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968df502-8141-41f5-b492-9ecefaa5450b admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:24.199223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968df502-8141-41f5-b492-9ecefaa5450b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:24.201273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968df502-8141-41f5-b492-9ecefaa5450b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:24.235208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 945/1898] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:24 2022] POST /identity/v3/projects => generated 390 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:24.240251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8c568-7802-4160-b84d-b9726e5571bd None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:24.256110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8c568-7802-4160-b84d-b9726e5571bd None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:24.341374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ad2d5-15b4-44de-9783-105063de6569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:24.342259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ad2d5-15b4-44de-9783-105063de6569 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:24.343105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ad2d5-15b4-44de-9783-105063de6569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:24.343684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ad2d5-15b4-44de-9783-105063de6569 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:24.344434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ad2d5-15b4-44de-9783-105063de6569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:24.347009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ad2d5-15b4-44de-9783-105063de6569 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:24.387425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 954/1899] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:24 2022] POST /identity/v3/users => generated 302 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:24.391509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968df502-8141-41f5-b492-9ecefaa5450b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:24.419476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968df502-8141-41f5-b492-9ecefaa5450b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:24.442078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfe679-2db9-440e-8186-1aa0ffc9f966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:24.443082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfe679-2db9-440e-8186-1aa0ffc9f966 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:24.443626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfe679-2db9-440e-8186-1aa0ffc9f966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:24.443626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfe679-2db9-440e-8186-1aa0ffc9f966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:24.459816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfe679-2db9-440e-8186-1aa0ffc9f966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:24.459816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfe679-2db9-440e-8186-1aa0ffc9f966 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:24.478489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 946/1900] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:24 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:24.481517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ad2d5-15b4-44de-9783-105063de6569 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:24.503175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ad2d5-15b4-44de-9783-105063de6569 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:24.509837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bf4de-018f-4573-b436-0e5b28485882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:24.511352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf4de-018f-4573-b436-0e5b28485882 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:24.512274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf4de-018f-4573-b436-0e5b28485882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:24.512960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf4de-018f-4573-b436-0e5b28485882 admin admin] PATH_INFO: `/v3/projects/983af3cac53f4f5a97358bab593d06fd/users/ab6d92cac3a64808be248d95c630782a/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:24.551135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:24.558302 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bf4de-018f-4573-b436-0e5b28485882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983af3cac53f4f5a97358bab593d06fd, user_id=ab6d92cac3a64808be248d95c630782a, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:24.563385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bf4de-018f-4573-b436-0e5b28485882 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:24.593907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 955/1901] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:24 2022] PUT /identity/v3/projects/983af3cac53f4f5a97358bab593d06fd/users/ab6d92cac3a64808be248d95c630782a/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:24.601624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfe679-2db9-440e-8186-1aa0ffc9f966 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:24.615359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfe679-2db9-440e-8186-1aa0ffc9f966 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:24.667285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bf4de-018f-4573-b436-0e5b28485882 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:24.669475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-642bf4de-018f-4573-b436-0e5b28485882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:24.719543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dd478-3fd2-440e-93ab-7b06ec8835bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:24.720451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dd478-3fd2-440e-93ab-7b06ec8835bc admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:24.720941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dd478-3fd2-440e-93ab-7b06ec8835bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:24.721740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dd478-3fd2-440e-93ab-7b06ec8835bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:24.722705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732dd478-3fd2-440e-93ab-7b06ec8835bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:24.732237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732dd478-3fd2-440e-93ab-7b06ec8835bc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:24.743492 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 947/1902] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:24 2022] GET /identity/v3/roles => generated 1427 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:24.747954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dd478-3fd2-440e-93ab-7b06ec8835bc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:24.767121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dd478-3fd2-440e-93ab-7b06ec8835bc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:24.773872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf5b07-d2e0-4166-9bfa-6eca22aecec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:24.774535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf5b07-d2e0-4166-9bfa-6eca22aecec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:24.774883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf5b07-d2e0-4166-9bfa-6eca22aecec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:24.775143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf5b07-d2e0-4166-9bfa-6eca22aecec3 admin admin] PATH_INFO: `/v3/projects/983af3cac53f4f5a97358bab593d06fd/users/ab6d92cac3a64808be248d95c630782a/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:24.783135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf5b07-d2e0-4166-9bfa-6eca22aecec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983af3cac53f4f5a97358bab593d06fd, user_id=ab6d92cac3a64808be248d95c630782a, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:24.785951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf5b07-d2e0-4166-9bfa-6eca22aecec3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:24.824668 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bf4de-018f-4573-b436-0e5b28485882 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:24.847368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 948/1903] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:24 2022] PUT /identity/v3/projects/983af3cac53f4f5a97358bab593d06fd/users/ab6d92cac3a64808be248d95c630782a/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:24.854063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b809-1b8d-4866-9bde-994ecae971c9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:24.854447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b809-1b8d-4866-9bde-994ecae971c9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:24.855099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b809-1b8d-4866-9bde-994ecae971c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:24.877862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:24.922695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c9e5b809-1b8d-4866-9bde-994ecae971c9 None None] MFA Rules not processed for user `ab6d92cac3a64808be248d95c630782a`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:24.922695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c9e5b809-1b8d-4866-9bde-994ecae971c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:24.938581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:24.939562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:24.940016 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:24.940438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] PATH_INFO: `/v3/users/8f4832ff60ba4d3283913f2b0c157ec2` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:24.943493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f4832ff60ba4d3283913f2b0c157ec2)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:24.946401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:25.033723 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f4832ff60ba4d3283913f2b0c157ec2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.053499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 949/1904] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:24 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:25.054718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5b809-1b8d-4866-9bde-994ecae971c9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:25.057285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c9e5b809-1b8d-4866-9bde-994ecae971c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:25.059334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f4832ff60ba4d3283913f2b0c157ec2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.068327 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f4832ff60ba4d3283913f2b0c157ec2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.077635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5b809-1b8d-4866-9bde-994ecae971c9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:25.079487 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f4832ff60ba4d3283913f2b0c157ec2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.080722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f4832ff60ba4d3283913f2b0c157ec2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.082661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:25.083541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:25.084049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:25.084184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] PATH_INFO: `/v3/users/6b02d925639a455c844929c6841088fb` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:25.086170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b02d925639a455c844929c6841088fb)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:25.089153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:25.090555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 956/1905] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:24 2022] DELETE /identity/v3/users/8f4832ff60ba4d3283913f2b0c157ec2 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:25.091656 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:25.094608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:25.130462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:25.156275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b02d925639a455c844929c6841088fb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.166217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b02d925639a455c844929c6841088fb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.167545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b02d925639a455c844929c6841088fb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.182870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b02d925639a455c844929c6841088fb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.184055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179fdcb-c153-4850-9f86-ca5a5f29545e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:25.188027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a602e2-1aff-438a-b8bc-6e2f719e90a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:25.188923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a602e2-1aff-438a-b8bc-6e2f719e90a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:25.189275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a602e2-1aff-438a-b8bc-6e2f719e90a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:25.189701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a602e2-1aff-438a-b8bc-6e2f719e90a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:25.191136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cdd449ca-e061-4a89-9f43-995d6fc40cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b02d925639a455c844929c6841088fb'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.191699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b3a602e2-1aff-438a-b8bc-6e2f719e90a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:25.203495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 950/1906] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:25 2022] DELETE /identity/v3/users/6b02d925639a455c844929c6841088fb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:25.205268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7aee-4168-4321-adfc-374ccd081136 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:25.205762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7aee-4168-4321-adfc-374ccd081136 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:25.206184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7aee-4168-4321-adfc-374ccd081136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:25.226935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e24b7aee-4168-4321-adfc-374ccd081136 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:25.228549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e24b7aee-4168-4321-adfc-374ccd081136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:25.267911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a602e2-1aff-438a-b8bc-6e2f719e90a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:25.269476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a602e2-1aff-438a-b8bc-6e2f719e90a0 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:25.281729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 957/1907] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:25.282715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09147d-e39b-4aed-a5f8-3216bd012267 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:25.283145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09147d-e39b-4aed-a5f8-3216bd012267 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:25.283426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09147d-e39b-4aed-a5f8-3216bd012267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:25.290455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:25.303733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 951/1908] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:25.311039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8f09147d-e39b-4aed-a5f8-3216bd012267 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:25.312264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8f09147d-e39b-4aed-a5f8-3216bd012267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:25.316833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7aee-4168-4321-adfc-374ccd081136 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:25.319239 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e24b7aee-4168-4321-adfc-374ccd081136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:25.324765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 958/1909] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:25.345050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09147d-e39b-4aed-a5f8-3216bd012267 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:25.346546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8f09147d-e39b-4aed-a5f8-3216bd012267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:25.419731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7aee-4168-4321-adfc-374ccd081136 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:25.424538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3fd65-fff5-4c80-a6a1-7a612ae2e993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:25.424971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3fd65-fff5-4c80-a6a1-7a612ae2e993 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:25.425350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3fd65-fff5-4c80-a6a1-7a612ae2e993 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:25.425711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3fd65-fff5-4c80-a6a1-7a612ae2e993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:25.443202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09147d-e39b-4aed-a5f8-3216bd012267 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:25.450986 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c79e8-107a-4e80-893a-fd47e258fb70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:25.451811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c79e8-107a-4e80-893a-fd47e258fb70 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:25.452212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c79e8-107a-4e80-893a-fd47e258fb70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:25.452553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c79e8-107a-4e80-893a-fd47e258fb70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:25.460417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3fd65-fff5-4c80-a6a1-7a612ae2e993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:25.464095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c79e8-107a-4e80-893a-fd47e258fb70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:25.465048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3fd65-fff5-4c80-a6a1-7a612ae2e993 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:25.465711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c79e8-107a-4e80-893a-fd47e258fb70 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:25.478548 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 959/1910] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:25.479673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c79e8-107a-4e80-893a-fd47e258fb70 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:25.481321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fa4c79e8-107a-4e80-893a-fd47e258fb70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:25.497764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c79e8-107a-4e80-893a-fd47e258fb70 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:25.501309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 952/1911] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:25.504117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:25.504853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:25.505618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:25.505618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] PATH_INFO: `/v3/users/1825ef17f0e14e2e8015043ad2316d82` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:25.507001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1825ef17f0e14e2e8015043ad2316d82)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:25.510115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:25.611552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1825ef17f0e14e2e8015043ad2316d82'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.627267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:25.637862 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1825ef17f0e14e2e8015043ad2316d82'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.651728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1825ef17f0e14e2e8015043ad2316d82'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.667696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1825ef17f0e14e2e8015043ad2316d82'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.668624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1825ef17f0e14e2e8015043ad2316d82'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:25.679839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 960/1912] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:25 2022] DELETE /identity/v3/users/1825ef17f0e14e2e8015043ad2316d82 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:25.686490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b69e5-4d9d-4659-a971-e607ffb78d47 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:25.686858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b69e5-4d9d-4659-a971-e607ffb78d47 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:25.687311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b69e5-4d9d-4659-a971-e607ffb78d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:25.715451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1a1b69e5-4d9d-4659-a971-e607ffb78d47 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:25.716995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1a1b69e5-4d9d-4659-a971-e607ffb78d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:25.814406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 953/1913] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:25.839219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:25.842999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:26.128677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c1971-f123-478c-8d5f-7ab7a34136b9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:26.144468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c9530-2f25-4eac-a394-78650782afb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:26.148128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c9530-2f25-4eac-a394-78650782afb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:26.148561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c9530-2f25-4eac-a394-78650782afb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:26.148956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c9530-2f25-4eac-a394-78650782afb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:26.192440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c9530-2f25-4eac-a394-78650782afb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:26.200542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c9530-2f25-4eac-a394-78650782afb1 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:26.247392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 961/1914] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:26.300059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b69e5-4d9d-4659-a971-e607ffb78d47 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:26.301672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1a1b69e5-4d9d-4659-a971-e607ffb78d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:26.326590 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b69e5-4d9d-4659-a971-e607ffb78d47 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:26.336983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:26.337927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:26.338213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:26.339471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] PATH_INFO: `/v3/projects/82d0a188969441129bd72a3477f2f8bd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:26.340997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82d0a188969441129bd72a3477f2f8bd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:26.343021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:26.481829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82d0a188969441129bd72a3477f2f8bd', 'request_id': 'req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.491080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82d0a188969441129bd72a3477f2f8bd', 'request_id': 'req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.509475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82d0a188969441129bd72a3477f2f8bd', 'request_id': 'req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.543516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] The token cache is being invalidate because project 82d0a188969441129bd72a3477f2f8bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:26.543829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.547532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.558950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 954/1915] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:26 2022] DELETE /identity/v3/projects/82d0a188969441129bd72a3477f2f8bd => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:26.562844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c9530-2f25-4eac-a394-78650782afb1 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:26.574693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:26.577162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:26.598982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c9530-2f25-4eac-a394-78650782afb1 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:26.655648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:26.715463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7d6c0-a2fd-4937-87f5-6b4ba658d8f5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:26.720546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:26.721219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:26.721541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:26.721854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] PATH_INFO: `/v3/projects/b4f9a398e4d54c3f9a1822fd2b2c125b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:26.723171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f9a398e4d54c3f9a1822fd2b2c125b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:26.725168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:26.730006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:26.731447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:26.731927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:26.732214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] PATH_INFO: `/v3/users/301303a6633c4e71ad4e2011495c4392` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:26.734560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=301303a6633c4e71ad4e2011495c4392)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:26.740481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:26.785528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f9a398e4d54c3f9a1822fd2b2c125b', 'request_id': 'req-77f00387-b8af-4519-8ee2-f98f4cfec154'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.794921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f9a398e4d54c3f9a1822fd2b2c125b', 'request_id': 'req-77f00387-b8af-4519-8ee2-f98f4cfec154'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.802114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f9a398e4d54c3f9a1822fd2b2c125b', 'request_id': 'req-77f00387-b8af-4519-8ee2-f98f4cfec154'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.817496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] The token cache is being invalidate because project b4f9a398e4d54c3f9a1822fd2b2c125b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:26.817978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.818662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.824247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 955/1916] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:26 2022] DELETE /identity/v3/projects/b4f9a398e4d54c3f9a1822fd2b2c125b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:26.825332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:26.826811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:26.850673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '301303a6633c4e71ad4e2011495c4392'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.866655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '301303a6633c4e71ad4e2011495c4392'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.878606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '301303a6633c4e71ad4e2011495c4392'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.891962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '301303a6633c4e71ad4e2011495c4392'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.892821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '301303a6633c4e71ad4e2011495c4392'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:26.895075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:26.904383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 962/1917] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:26 2022] DELETE /identity/v3/users/301303a6633c4e71ad4e2011495c4392 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:26.916550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f00387-b8af-4519-8ee2-f98f4cfec154 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:26.963012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:26.965829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:27.016646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:27.017427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:27.017734 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:27.018144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] PATH_INFO: `/v3/users/25f53d03ba0b4ae6a263d031a750f114` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:27.020410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f53d03ba0b4ae6a263d031a750f114)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:27.023614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:27.032279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:27.110517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaae5df-f0df-4c5f-8951-f0cb15b1c0ec admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:27.116413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:27.117075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:27.117403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:27.117740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] PATH_INFO: `/v3/projects/a93b6a1166cd4da9b6bd53cc5731c851` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:27.119103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f53d03ba0b4ae6a263d031a750f114'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.119454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93b6a1166cd4da9b6bd53cc5731c851)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:27.121522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:27.131301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f53d03ba0b4ae6a263d031a750f114'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.132053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f53d03ba0b4ae6a263d031a750f114'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.149226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f53d03ba0b4ae6a263d031a750f114'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.168423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f53d03ba0b4ae6a263d031a750f114'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.183762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 956/1918] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:26 2022] DELETE /identity/v3/users/25f53d03ba0b4ae6a263d031a750f114 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:27.210043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93b6a1166cd4da9b6bd53cc5731c851', 'request_id': 'req-3375ca09-a8c4-446e-a797-f24da91a0195'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.221204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93b6a1166cd4da9b6bd53cc5731c851', 'request_id': 'req-3375ca09-a8c4-446e-a797-f24da91a0195'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.234401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93b6a1166cd4da9b6bd53cc5731c851', 'request_id': 'req-3375ca09-a8c4-446e-a797-f24da91a0195'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.250245 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] The token cache is being invalidate because project a93b6a1166cd4da9b6bd53cc5731c851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:27.251650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.253114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.256692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 963/1919] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:26 2022] DELETE /identity/v3/projects/a93b6a1166cd4da9b6bd53cc5731c851 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:27.262028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:27.263569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:27.313836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:27.316258 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:27.394682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf91717-924f-4f1c-827f-2897400c43c0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:27.401018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:27.402055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:27.402481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:27.403012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] PATH_INFO: `/v3/users/ab6d92cac3a64808be248d95c630782a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:27.404948 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab6d92cac3a64808be248d95c630782a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:27.410374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:27.432303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375ca09-a8c4-446e-a797-f24da91a0195 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:27.437739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:27.438517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:27.438915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:27.439296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] PATH_INFO: `/v3/projects/150f5a45d1624162b377a4a34a3bad9b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:27.440545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=150f5a45d1624162b377a4a34a3bad9b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:27.442612 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:27.471909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab6d92cac3a64808be248d95c630782a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.493368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab6d92cac3a64808be248d95c630782a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.496799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab6d92cac3a64808be248d95c630782a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.515377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:27.532541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab6d92cac3a64808be248d95c630782a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.543402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '150f5a45d1624162b377a4a34a3bad9b', 'request_id': 'req-458d2613-de0f-4051-a820-e84e5bcba310'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.549729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab6d92cac3a64808be248d95c630782a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.554025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '150f5a45d1624162b377a4a34a3bad9b', 'request_id': 'req-458d2613-de0f-4051-a820-e84e5bcba310'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.564168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 957/1920] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:27 2022] DELETE /identity/v3/users/ab6d92cac3a64808be248d95c630782a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:27.564842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '150f5a45d1624162b377a4a34a3bad9b', 'request_id': 'req-458d2613-de0f-4051-a820-e84e5bcba310'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.583908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] The token cache is being invalidate because project 150f5a45d1624162b377a4a34a3bad9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:27.584402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.585184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.589198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 964/1921] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:27 2022] DELETE /identity/v3/projects/150f5a45d1624162b377a4a34a3bad9b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:27.593430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:27.595452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:27.601117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:27.712220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:27.713687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:27.733203 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb83b4e-ac56-4b51-8148-0165b892187b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:27.759021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:27.760126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:27.760637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:27.761830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] PATH_INFO: `/v3/users/5fb089ba65f64e0ba3df9e84f0179e70` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:27.763509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb089ba65f64e0ba3df9e84f0179e70)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:27.766166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:27.820005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d2613-de0f-4051-a820-e84e5bcba310 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:27.825545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:27.826260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:27.826543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:27.827325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] PATH_INFO: `/v3/projects/604f1f42e4af4119be09ff3dd286d8be` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:27.828519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=604f1f42e4af4119be09ff3dd286d8be)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:27.831377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:27.845676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb089ba65f64e0ba3df9e84f0179e70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.856521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb089ba65f64e0ba3df9e84f0179e70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.857558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb089ba65f64e0ba3df9e84f0179e70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.879455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb089ba65f64e0ba3df9e84f0179e70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.889262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb089ba65f64e0ba3df9e84f0179e70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.903695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '604f1f42e4af4119be09ff3dd286d8be', 'request_id': 'req-f59081bc-2156-424e-8e0f-439cd6c47710'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.906980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 958/1922] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:27 2022] DELETE /identity/v3/users/5fb089ba65f64e0ba3df9e84f0179e70 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:27.921499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '604f1f42e4af4119be09ff3dd286d8be', 'request_id': 'req-f59081bc-2156-424e-8e0f-439cd6c47710'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.931107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '604f1f42e4af4119be09ff3dd286d8be', 'request_id': 'req-f59081bc-2156-424e-8e0f-439cd6c47710'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.948358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:27.948509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] The token cache is being invalidate because project 604f1f42e4af4119be09ff3dd286d8be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:27.948681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.949431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f59081bc-2156-424e-8e0f-439cd6c47710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:27.950103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:27.951167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 965/1923] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:27 2022] DELETE /identity/v3/projects/604f1f42e4af4119be09ff3dd286d8be => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:27.956124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f4633-f535-422a-bf9e-54f9db99e344 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:27.956416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f4633-f535-422a-bf9e-54f9db99e344 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:27.956744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f4633-f535-422a-bf9e-54f9db99e344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:27.998867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b23f4633-f535-422a-bf9e-54f9db99e344 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:28.000241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b23f4633-f535-422a-bf9e-54f9db99e344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.049229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43b7fd-e33f-4523-bc29-e4b8aa11c60b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.054245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.054663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:28.054893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.055246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.055424 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] PATH_INFO: `/v3/projects/983af3cac53f4f5a97358bab593d06fd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.056881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=983af3cac53f4f5a97358bab593d06fd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.059068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.075263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 966/1924] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:28.083033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eefe5b-1588-45d6-802c-16e9f1860c74 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.084166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eefe5b-1588-45d6-802c-16e9f1860c74 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.084609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eefe5b-1588-45d6-802c-16e9f1860c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.089989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:28.116624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-48eefe5b-1588-45d6-802c-16e9f1860c74 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:28.117877 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-48eefe5b-1588-45d6-802c-16e9f1860c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.124856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983af3cac53f4f5a97358bab593d06fd', 'request_id': 'req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.132495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983af3cac53f4f5a97358bab593d06fd', 'request_id': 'req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.139128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983af3cac53f4f5a97358bab593d06fd', 'request_id': 'req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.154478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] The token cache is being invalidate because project 983af3cac53f4f5a97358bab593d06fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:28.154682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.155324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f2a80b42-289c-47fa-9c16-9e0c7c39fe49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.162738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 959/1925] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:27 2022] DELETE /identity/v3/projects/983af3cac53f4f5a97358bab593d06fd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:28.166146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35b4ec-1822-4ab3-ab7c-521dcd24f219 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.166659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35b4ec-1822-4ab3-ab7c-521dcd24f219 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.167175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35b4ec-1822-4ab3-ab7c-521dcd24f219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.193341 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ce35b4ec-1822-4ab3-ab7c-521dcd24f219 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:28.194592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ce35b4ec-1822-4ab3-ab7c-521dcd24f219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.201491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 967/1926] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:28.204068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 960/1927] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:28.207038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eefe5b-1588-45d6-802c-16e9f1860c74 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.211489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e5f29-1394-4142-980c-afbec73c5b58 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.212162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e5f29-1394-4142-980c-afbec73c5b58 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.213342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e5f29-1394-4142-980c-afbec73c5b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.217152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eefe5b-1588-45d6-802c-16e9f1860c74 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.222399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c7fda-e836-410e-b74c-1645bdcd9304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.223037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c7fda-e836-410e-b74c-1645bdcd9304 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.223407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c7fda-e836-410e-b74c-1645bdcd9304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.223684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c7fda-e836-410e-b74c-1645bdcd9304 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.224340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c7fda-e836-410e-b74c-1645bdcd9304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:28.224647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c7fda-e836-410e-b74c-1645bdcd9304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.226298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c7fda-e836-410e-b74c-1645bdcd9304 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.238792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-432e5f29-1394-4142-980c-afbec73c5b58 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:28.240169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-432e5f29-1394-4142-980c-afbec73c5b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.241002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 968/1928] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:28.246108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c7fda-e836-410e-b74c-1645bdcd9304 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.254959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 961/1929] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:28.255867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e5f29-1394-4142-980c-afbec73c5b58 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.257233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-432e5f29-1394-4142-980c-afbec73c5b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.257344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c7fda-e836-410e-b74c-1645bdcd9304 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.262940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ca124-c013-4403-bdbc-925342494152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.263564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ca124-c013-4403-bdbc-925342494152 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.263845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ca124-c013-4403-bdbc-925342494152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.264123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ca124-c013-4403-bdbc-925342494152 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.269692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ca124-c013-4403-bdbc-925342494152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.271814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ca124-c013-4403-bdbc-925342494152 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.277749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e5f29-1394-4142-980c-afbec73c5b58 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.283355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.284030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.284308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.284580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] PATH_INFO: `/v3/projects/dc92bc3246394aec9dfe9d4673c77dfb` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.287447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc92bc3246394aec9dfe9d4673c77dfb)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.289603 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.311609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 969/1930] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/projects => generated 386 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:28.311609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ca124-c013-4403-bdbc-925342494152 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.357113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc92bc3246394aec9dfe9d4673c77dfb', 'request_id': 'req-dae039ab-9d35-41e3-8259-bfc2608a6393'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.366572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc92bc3246394aec9dfe9d4673c77dfb', 'request_id': 'req-dae039ab-9d35-41e3-8259-bfc2608a6393'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.377149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc92bc3246394aec9dfe9d4673c77dfb', 'request_id': 'req-dae039ab-9d35-41e3-8259-bfc2608a6393'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.388142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ca124-c013-4403-bdbc-925342494152 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.392472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] The token cache is being invalidate because project dc92bc3246394aec9dfe9d4673c77dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:28.393008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.393808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:28.395663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 962/1931] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:28 2022] DELETE /identity/v3/projects/dc92bc3246394aec9dfe9d4673c77dfb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:28.396391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.397792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.469306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf835849-3515-4970-9224-f18bebe877bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.470128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf835849-3515-4970-9224-f18bebe877bb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.470474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf835849-3515-4970-9224-f18bebe877bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.470816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf835849-3515-4970-9224-f18bebe877bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.471541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf835849-3515-4970-9224-f18bebe877bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:28.471936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf835849-3515-4970-9224-f18bebe877bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.474318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf835849-3515-4970-9224-f18bebe877bb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.477304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:28.480430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae039ab-9d35-41e3-8259-bfc2608a6393 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.489108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 970/1932] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:28.490824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b5456-184c-4cc3-b22e-b58ccf7588eb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.491197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b5456-184c-4cc3-b22e-b58ccf7588eb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.491544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b5456-184c-4cc3-b22e-b58ccf7588eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.493437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4bd64-dae1-4d5c-b29c-238f6be8e9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.494695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4bd64-dae1-4d5c-b29c-238f6be8e9de admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.495380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4bd64-dae1-4d5c-b29c-238f6be8e9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.495798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4bd64-dae1-4d5c-b29c-238f6be8e9de admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.497281 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4bd64-dae1-4d5c-b29c-238f6be8e9de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.499548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4bd64-dae1-4d5c-b29c-238f6be8e9de admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.515091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-951b5456-184c-4cc3-b22e-b58ccf7588eb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:28.516499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-951b5456-184c-4cc3-b22e-b58ccf7588eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.525955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 971/1933] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:28.526425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 963/1934] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:28.527951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b5456-184c-4cc3-b22e-b58ccf7588eb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.530552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-951b5456-184c-4cc3-b22e-b58ccf7588eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.537037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4bd64-dae1-4d5c-b29c-238f6be8e9de admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.543577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b5456-184c-4cc3-b22e-b58ccf7588eb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.548341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7889d-1040-45e5-b4f9-d3beb5019756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.548566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4bd64-dae1-4d5c-b29c-238f6be8e9de admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.548966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7889d-1040-45e5-b4f9-d3beb5019756 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.549230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7889d-1040-45e5-b4f9-d3beb5019756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.549567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7889d-1040-45e5-b4f9-d3beb5019756 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.550159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7889d-1040-45e5-b4f9-d3beb5019756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.554505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7889d-1040-45e5-b4f9-d3beb5019756 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.556663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560056-d6d1-4f1e-a402-48e25408d03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.557668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560056-d6d1-4f1e-a402-48e25408d03d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.558132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560056-d6d1-4f1e-a402-48e25408d03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.558570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560056-d6d1-4f1e-a402-48e25408d03d admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.559616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c560056-d6d1-4f1e-a402-48e25408d03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.562403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c560056-d6d1-4f1e-a402-48e25408d03d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.571011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=82,keystone:DELETE=13,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:28.589857 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 964/1935] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:28 2022] GET /identity/v3/roles => generated 1427 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:28.592320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16c06e-74c7-4af1-a69a-589757f6d366 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.592879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16c06e-74c7-4af1-a69a-589757f6d366 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.593167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16c06e-74c7-4af1-a69a-589757f6d366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.602453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 972/1936] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/projects => generated 374 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:28.603955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7889d-1040-45e5-b4f9-d3beb5019756 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.620306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2a16c06e-74c7-4af1-a69a-589757f6d366 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:28.621691 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2a16c06e-74c7-4af1-a69a-589757f6d366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.672783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7889d-1040-45e5-b4f9-d3beb5019756 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.677515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0ae45-e563-45ab-bb29-c8bbb6f5fa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.678148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0ae45-e563-45ab-bb29-c8bbb6f5fa28 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.678422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0ae45-e563-45ab-bb29-c8bbb6f5fa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.678691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0ae45-e563-45ab-bb29-c8bbb6f5fa28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.679327 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0ae45-e563-45ab-bb29-c8bbb6f5fa28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.681001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0ae45-e563-45ab-bb29-c8bbb6f5fa28 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.691264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 965/1937] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:28.691679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 973/1938] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:28 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:28.692098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16c06e-74c7-4af1-a69a-589757f6d366 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.695627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0ae45-e563-45ab-bb29-c8bbb6f5fa28 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.702069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16c06e-74c7-4af1-a69a-589757f6d366 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.706938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0ae45-e563-45ab-bb29-c8bbb6f5fa28 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.707896 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf39d5-3c6d-4bd0-bfc3-fb25b84a125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.708649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf39d5-3c6d-4bd0-bfc3-fb25b84a125f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.708934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf39d5-3c6d-4bd0-bfc3-fb25b84a125f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.709209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf39d5-3c6d-4bd0-bfc3-fb25b84a125f admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.709905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf39d5-3c6d-4bd0-bfc3-fb25b84a125f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.712634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf39d5-3c6d-4bd0-bfc3-fb25b84a125f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.713194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ece8a1-588d-4942-8328-50166c960676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.714050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ece8a1-588d-4942-8328-50166c960676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.714382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ece8a1-588d-4942-8328-50166c960676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.714663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ece8a1-588d-4942-8328-50166c960676 admin admin] PATH_INFO: `/v3/projects/7a48ae1d72bc4fbc9c7022961ac9b705/users/d01b0a4161244f278e6dfac30a5bb3a3/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.731589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ece8a1-588d-4942-8328-50166c960676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a48ae1d72bc4fbc9c7022961ac9b705, user_id=d01b0a4161244f278e6dfac30a5bb3a3, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.735631 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ece8a1-588d-4942-8328-50166c960676 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.738936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 966/1939] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:28.743417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf39d5-3c6d-4bd0-bfc3-fb25b84a125f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.750133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 974/1940] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:28 2022] PUT /identity/v3/projects/7a48ae1d72bc4fbc9c7022961ac9b705/users/d01b0a4161244f278e6dfac30a5bb3a3/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:28.760387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ece8a1-588d-4942-8328-50166c960676 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.760493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf39d5-3c6d-4bd0-bfc3-fb25b84a125f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.813637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e39698-2598-40d2-85cd-07b625040426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.814266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e39698-2598-40d2-85cd-07b625040426 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.814548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e39698-2598-40d2-85cd-07b625040426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.814820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e39698-2598-40d2-85cd-07b625040426 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.815171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ece8a1-588d-4942-8328-50166c960676 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.815452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e39698-2598-40d2-85cd-07b625040426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:28.815839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e39698-2598-40d2-85cd-07b625040426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.817800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e39698-2598-40d2-85cd-07b625040426 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.820000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01590e35-ad2d-408c-b17c-726302bdde7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.820615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01590e35-ad2d-408c-b17c-726302bdde7f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.821157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01590e35-ad2d-408c-b17c-726302bdde7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.821157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01590e35-ad2d-408c-b17c-726302bdde7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.821861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01590e35-ad2d-408c-b17c-726302bdde7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.823459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01590e35-ad2d-408c-b17c-726302bdde7f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.830431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 967/1941] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:28.831651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e955-5a4d-4391-853e-3e1aac364f13 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.831992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e955-5a4d-4391-853e-3e1aac364f13 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.832263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e955-5a4d-4391-853e-3e1aac364f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.836573 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 975/1942] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:28 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:28.836573 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01590e35-ad2d-408c-b17c-726302bdde7f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.845548 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01590e35-ad2d-408c-b17c-726302bdde7f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.850749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24d8b-31ad-4272-90bb-5850742bbab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.851351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24d8b-31ad-4272-90bb-5850742bbab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.851699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24d8b-31ad-4272-90bb-5850742bbab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.851979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24d8b-31ad-4272-90bb-5850742bbab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.852741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea24d8b-31ad-4272-90bb-5850742bbab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.855312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea24d8b-31ad-4272-90bb-5850742bbab6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:28.860647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dd05e955-5a4d-4391-853e-3e1aac364f13 None None] MFA Rules not processed for user `d01b0a4161244f278e6dfac30a5bb3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:28.861865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dd05e955-5a4d-4391-853e-3e1aac364f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:28.889762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 976/1943] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/projects => generated 368 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:28.889762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24d8b-31ad-4272-90bb-5850742bbab6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.907768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:28.944966 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 968/1944] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:28.946221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05e955-5a4d-4391-853e-3e1aac364f13 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:28.961301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05e955-5a4d-4391-853e-3e1aac364f13 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.966440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:28.977683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24d8b-31ad-4272-90bb-5850742bbab6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:28.983750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83957c-c495-41cd-99d2-e1ba9894ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:28.984406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83957c-c495-41cd-99d2-e1ba9894ff00 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:28.984750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83957c-c495-41cd-99d2-e1ba9894ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:28.985035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83957c-c495-41cd-99d2-e1ba9894ff00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:28.986261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83957c-c495-41cd-99d2-e1ba9894ff00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:28.988888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83957c-c495-41cd-99d2-e1ba9894ff00 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.000423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 977/1945] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:28 2022] GET /identity/v3/roles => generated 1427 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:29.001536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83957c-c495-41cd-99d2-e1ba9894ff00 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.015184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83957c-c495-41cd-99d2-e1ba9894ff00 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.028149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3527047-1144-462a-be2d-d2b164be4dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.028675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3527047-1144-462a-be2d-d2b164be4dfb admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.028954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3527047-1144-462a-be2d-d2b164be4dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.029249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3527047-1144-462a-be2d-d2b164be4dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.029882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3527047-1144-462a-be2d-d2b164be4dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.031957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3527047-1144-462a-be2d-d2b164be4dfb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.034662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881c3a4-2074-4eed-80e3-5c0ec9bdd30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.035290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881c3a4-2074-4eed-80e3-5c0ec9bdd30a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.035579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881c3a4-2074-4eed-80e3-5c0ec9bdd30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.035878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881c3a4-2074-4eed-80e3-5c0ec9bdd30a admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.036780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881c3a4-2074-4eed-80e3-5c0ec9bdd30a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.039288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881c3a4-2074-4eed-80e3-5c0ec9bdd30a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.064094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 978/1946] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:29 2022] POST /identity/v3/projects => generated 388 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:29.065190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3527047-1144-462a-be2d-d2b164be4dfb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.070635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 969/1947] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:28 2022] POST /identity/v3/users => generated 285 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:29.072099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881c3a4-2074-4eed-80e3-5c0ec9bdd30a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.086181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881c3a4-2074-4eed-80e3-5c0ec9bdd30a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.162952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3527047-1144-462a-be2d-d2b164be4dfb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.164021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22259d5-e52f-48f5-97d3-9bd3df15abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.164640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22259d5-e52f-48f5-97d3-9bd3df15abf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.164925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22259d5-e52f-48f5-97d3-9bd3df15abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.165435 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22259d5-e52f-48f5-97d3-9bd3df15abf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.166326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22259d5-e52f-48f5-97d3-9bd3df15abf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.167715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03e486-d2f1-4d99-958b-289730546915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.168186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22259d5-e52f-48f5-97d3-9bd3df15abf2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.168374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03e486-d2f1-4d99-958b-289730546915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.168794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03e486-d2f1-4d99-958b-289730546915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.168910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03e486-d2f1-4d99-958b-289730546915 admin admin] PATH_INFO: `/v3/projects/a8dfc0cd97964af4a9c60c54a6ecc4b8/users/95c0197f338e4632a06c3cfea55422e4/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.185601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03e486-d2f1-4d99-958b-289730546915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8dfc0cd97964af4a9c60c54a6ecc4b8, user_id=95c0197f338e4632a06c3cfea55422e4, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.188708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03e486-d2f1-4d99-958b-289730546915 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.191426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 970/1948] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:29 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:29.192232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22259d5-e52f-48f5-97d3-9bd3df15abf2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.207763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 979/1949] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:29 2022] PUT /identity/v3/projects/a8dfc0cd97964af4a9c60c54a6ecc4b8/users/95c0197f338e4632a06c3cfea55422e4/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:29.217916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03e486-d2f1-4d99-958b-289730546915 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.218011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22259d5-e52f-48f5-97d3-9bd3df15abf2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.326745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03e486-d2f1-4d99-958b-289730546915 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.336454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc9e25-4331-4761-a8db-688a5b623195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.338490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc9e25-4331-4761-a8db-688a5b623195 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.338913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc9e25-4331-4761-a8db-688a5b623195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.339448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc9e25-4331-4761-a8db-688a5b623195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.340342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc9e25-4331-4761-a8db-688a5b623195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.342727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc9e25-4331-4761-a8db-688a5b623195 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.345708 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48529d0a-ea46-4e09-85dc-ae9dfc169f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.347813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48529d0a-ea46-4e09-85dc-ae9dfc169f37 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.348460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48529d0a-ea46-4e09-85dc-ae9dfc169f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.349952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48529d0a-ea46-4e09-85dc-ae9dfc169f37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.362221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 980/1950] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:29 2022] GET /identity/v3/roles => generated 1427 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:29.362427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48529d0a-ea46-4e09-85dc-ae9dfc169f37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.362521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5965b-106f-496b-8ad6-0206073ca97f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.362521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5965b-106f-496b-8ad6-0206073ca97f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.362521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5965b-106f-496b-8ad6-0206073ca97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.363323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48529d0a-ea46-4e09-85dc-ae9dfc169f37 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.383504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:29.395540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 971/1951] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:29 2022] GET /identity/v3/roles => generated 1427 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:29.398202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48529d0a-ea46-4e09-85dc-ae9dfc169f37 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.407715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-34f5965b-106f-496b-8ad6-0206073ca97f None None] MFA Rules not processed for user `95c0197f338e4632a06c3cfea55422e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:29.408969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34f5965b-106f-496b-8ad6-0206073ca97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:29.414677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48529d0a-ea46-4e09-85dc-ae9dfc169f37 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.430485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0353ae-cba1-4c90-b57f-d3cbf097c788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.433055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0353ae-cba1-4c90-b57f-d3cbf097c788 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.433340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0353ae-cba1-4c90-b57f-d3cbf097c788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.435806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0353ae-cba1-4c90-b57f-d3cbf097c788 admin admin] PATH_INFO: `/v3/projects/963c022af5584371bddd8690ce7ed097/users/ad52c43ccdc442abbb551fceb3d017f7/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.469556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0353ae-cba1-4c90-b57f-d3cbf097c788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=963c022af5584371bddd8690ce7ed097, user_id=ad52c43ccdc442abbb551fceb3d017f7, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.473410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0353ae-cba1-4c90-b57f-d3cbf097c788 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.491686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:29.505768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 981/1952] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:29.507287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5965b-106f-496b-8ad6-0206073ca97f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.509494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 972/1953] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:29 2022] PUT /identity/v3/projects/963c022af5584371bddd8690ce7ed097/users/ad52c43ccdc442abbb551fceb3d017f7/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:29.518769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0353ae-cba1-4c90-b57f-d3cbf097c788 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.523233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5965b-106f-496b-8ad6-0206073ca97f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.593638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff717ef-539a-485b-a8fd-c0736d42bde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.594259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff717ef-539a-485b-a8fd-c0736d42bde4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.594535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff717ef-539a-485b-a8fd-c0736d42bde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.594807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff717ef-539a-485b-a8fd-c0736d42bde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.595445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff717ef-539a-485b-a8fd-c0736d42bde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.597050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff717ef-539a-485b-a8fd-c0736d42bde4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.599319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0353ae-cba1-4c90-b57f-d3cbf097c788 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.604427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aa812-8373-434d-91cd-22472b1bd781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.605112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aa812-8373-434d-91cd-22472b1bd781 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.605399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aa812-8373-434d-91cd-22472b1bd781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.605723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aa812-8373-434d-91cd-22472b1bd781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.606374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aa812-8373-434d-91cd-22472b1bd781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.607975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aa812-8373-434d-91cd-22472b1bd781 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.608433 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 982/1954] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:29 2022] GET /identity/v3/roles => generated 1427 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:29.609414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff717ef-539a-485b-a8fd-c0736d42bde4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.611047 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aff717ef-539a-485b-a8fd-c0736d42bde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:29.624373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 973/1955] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:29 2022] GET /identity/v3/roles => generated 1427 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:29.625106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aa812-8373-434d-91cd-22472b1bd781 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.636361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aa812-8373-434d-91cd-22472b1bd781 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.643565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d3297-a99f-4d47-88f7-644a0f832612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.644247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d3297-a99f-4d47-88f7-644a0f832612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.644522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d3297-a99f-4d47-88f7-644a0f832612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.644798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d3297-a99f-4d47-88f7-644a0f832612 admin admin] PATH_INFO: `/v3/projects/afdce98bead34925b8192e2f89b9f699/users/85101a4497e94478a64800b06740affd/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.684360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d3297-a99f-4d47-88f7-644a0f832612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afdce98bead34925b8192e2f89b9f699, user_id=85101a4497e94478a64800b06740affd, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.687184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d3297-a99f-4d47-88f7-644a0f832612 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.688728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff717ef-539a-485b-a8fd-c0736d42bde4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.692640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d407738-7b40-410e-9bcb-c5dfe97f50a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.693260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d407738-7b40-410e-9bcb-c5dfe97f50a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.693543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d407738-7b40-410e-9bcb-c5dfe97f50a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.693853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d407738-7b40-410e-9bcb-c5dfe97f50a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.705253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 974/1956] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:29 2022] PUT /identity/v3/projects/afdce98bead34925b8192e2f89b9f699/users/85101a4497e94478a64800b06740affd/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:29.706091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d3297-a99f-4d47-88f7-644a0f832612 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.757707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:29.778484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d407738-7b40-410e-9bcb-c5dfe97f50a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.780393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d407738-7b40-410e-9bcb-c5dfe97f50a9 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.792643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 983/1957] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:29.793539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d3297-a99f-4d47-88f7-644a0f832612 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.794244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19f125-031e-4133-9fca-8c6a1dd87396 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.794820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19f125-031e-4133-9fca-8c6a1dd87396 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.795507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19f125-031e-4133-9fca-8c6a1dd87396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.799276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b234e14-7623-4e81-b254-bb5b6b062d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.799911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234e14-7623-4e81-b254-bb5b6b062d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.800197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234e14-7623-4e81-b254-bb5b6b062d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.800471 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234e14-7623-4e81-b254-bb5b6b062d51 admin admin] PATH_INFO: `/v3/projects/963c022af5584371bddd8690ce7ed097/users/ad52c43ccdc442abbb551fceb3d017f7/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.803491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b234e14-7623-4e81-b254-bb5b6b062d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=963c022af5584371bddd8690ce7ed097, user_id=ad52c43ccdc442abbb551fceb3d017f7, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.807676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b234e14-7623-4e81-b254-bb5b6b062d51 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.822983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 975/1958] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:29 2022] PUT /identity/v3/projects/963c022af5584371bddd8690ce7ed097/users/ad52c43ccdc442abbb551fceb3d017f7/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:29.824519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b234e14-7623-4e81-b254-bb5b6b062d51 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:29.843313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-eb19f125-031e-4133-9fca-8c6a1dd87396 None None] MFA Rules not processed for user `85101a4497e94478a64800b06740affd`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:29.845363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eb19f125-031e-4133-9fca-8c6a1dd87396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:29.905259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b234e14-7623-4e81-b254-bb5b6b062d51 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:29.910740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:29.911659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.911950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.912236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] PATH_INFO: `/v3/users/95c0197f338e4632a06c3cfea55422e4` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.913490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c0197f338e4632a06c3cfea55422e4)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:29.915632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:29.921812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:29.944342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 984/1959] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:29.947571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1071906-628f-47d0-ad51-feb93f7b0e31 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:29.948051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1071906-628f-47d0-ad51-feb93f7b0e31 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:29.948342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1071906-628f-47d0-ad51-feb93f7b0e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:29.984168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c0197f338e4632a06c3cfea55422e4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:29.985801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b1071906-628f-47d0-ad51-feb93f7b0e31 None None] MFA Rules not processed for user `ad52c43ccdc442abbb551fceb3d017f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:29.987453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b1071906-628f-47d0-ad51-feb93f7b0e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:29.994538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c0197f338e4632a06c3cfea55422e4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:29.995667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c0197f338e4632a06c3cfea55422e4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:30.019178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c0197f338e4632a06c3cfea55422e4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:30.030338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c0197f338e4632a06c3cfea55422e4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:30.044750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 976/1960] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:29 2022] DELETE /identity/v3/users/95c0197f338e4632a06c3cfea55422e4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:30.044750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:30.044750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:30.092475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 985/1961] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:29 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:30.092475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f786f-9db3-4dae-b3ed-416dea3a5d39 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:30.092475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f786f-9db3-4dae-b3ed-416dea3a5d39 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:30.092475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f786f-9db3-4dae-b3ed-416dea3a5d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:30.125300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cc7f786f-9db3-4dae-b3ed-416dea3a5d39 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:30.126570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cc7f786f-9db3-4dae-b3ed-416dea3a5d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:30.146208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9e5e3-ca02-4d8f-9c8a-260e7a6ce20d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:30.150715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0891d81e6e24ad99a9b532059c8c2af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:30.151468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:30.151760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:30.151929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:30.153914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:30.229920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 986/1962] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:30.231636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f786f-9db3-4dae-b3ed-416dea3a5d39 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:30.234208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cc7f786f-9db3-4dae-b3ed-416dea3a5d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:30.235073 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:30.238879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:30.241202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:30.251037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 977/1963] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:30.252057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:30.253437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:30.288853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:30.345930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f786f-9db3-4dae-b3ed-416dea3a5d39 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:30.351604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7c0e0-f57b-45e0-9d39-3dfc580e42bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:30.352543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7c0e0-f57b-45e0-9d39-3dfc580e42bb service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:30.353041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7c0e0-f57b-45e0-9d39-3dfc580e42bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:30.353498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7c0e0-f57b-45e0-9d39-3dfc580e42bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:30.389888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7ecd-81e2-4423-8209-50eb3233fd06 service glance] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:30.397423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82673d58-4043-4c24-bf41-3e82282e78f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:30.398981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82673d58-4043-4c24-bf41-3e82282e78f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:30.399435 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82673d58-4043-4c24-bf41-3e82282e78f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:30.401368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82673d58-4043-4c24-bf41-3e82282e78f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:30.438521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82673d58-4043-4c24-bf41-3e82282e78f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:30.440011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82673d58-4043-4c24-bf41-3e82282e78f3 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:30.440631 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7c0e0-f57b-45e0-9d39-3dfc580e42bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:30.441817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7c0e0-f57b-45e0-9d39-3dfc580e42bb service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:30.454342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 987/1965] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:30 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:30.460781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 978/1965] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:30.572888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7c0e0-f57b-45e0-9d39-3dfc580e42bb service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:30.579335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5ae7c0e0-f57b-45e0-9d39-3dfc580e42bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:30.600369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7c0e0-f57b-45e0-9d39-3dfc580e42bb service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:30.607805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:30.608461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:30.608875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:30.609161 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] PATH_INFO: `/v3/users/d01b0a4161244f278e6dfac30a5bb3a3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:30.610634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d01b0a4161244f278e6dfac30a5bb3a3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:30.612808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:30.696121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd01b0a4161244f278e6dfac30a5bb3a3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:30.720722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd01b0a4161244f278e6dfac30a5bb3a3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:30.734726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd01b0a4161244f278e6dfac30a5bb3a3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:30.747728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd01b0a4161244f278e6dfac30a5bb3a3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:30.748515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd01b0a4161244f278e6dfac30a5bb3a3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:30.762783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 988/1966] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:30 2022] DELETE /identity/v3/users/d01b0a4161244f278e6dfac30a5bb3a3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:30.763320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72fc1b-2e31-4a15-b988-a84cb00411db None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:30.763810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72fc1b-2e31-4a15-b988-a84cb00411db None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:30.763912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72fc1b-2e31-4a15-b988-a84cb00411db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:30.793347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ab72fc1b-2e31-4a15-b988-a84cb00411db None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:30.794910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ab72fc1b-2e31-4a15-b988-a84cb00411db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:30.876956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:30.879568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:30.942952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 979/1967] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:30.947527 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:30.951654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72fc1b-2e31-4a15-b988-a84cb00411db None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:30.954847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ab72fc1b-2e31-4a15-b988-a84cb00411db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:31.004497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1ae7d-9ef5-40a3-a2a3-ee0cd16080c7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.010172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.010815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.011116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.011398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] PATH_INFO: `/v3/projects/a8dfc0cd97964af4a9c60c54a6ecc4b8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.016664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8dfc0cd97964af4a9c60c54a6ecc4b8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.019511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:31.059468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72fc1b-2e31-4a15-b988-a84cb00411db None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.067007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79239f4f-0eeb-4e90-b5c5-ae6397a42f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.067647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79239f4f-0eeb-4e90-b5c5-ae6397a42f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.067978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79239f4f-0eeb-4e90-b5c5-ae6397a42f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.068259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79239f4f-0eeb-4e90-b5c5-ae6397a42f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.083833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79239f4f-0eeb-4e90-b5c5-ae6397a42f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.085268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79239f4f-0eeb-4e90-b5c5-ae6397a42f0d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:31.096566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8dfc0cd97964af4a9c60c54a6ecc4b8', 'request_id': 'req-3ec011a6-6335-4572-8147-607024e03055'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.097549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 980/1968] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:31.105002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8dfc0cd97964af4a9c60c54a6ecc4b8', 'request_id': 'req-3ec011a6-6335-4572-8147-607024e03055'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.111785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8dfc0cd97964af4a9c60c54a6ecc4b8', 'request_id': 'req-3ec011a6-6335-4572-8147-607024e03055'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.130956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] The token cache is being invalidate because project a8dfc0cd97964af4a9c60c54a6ecc4b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:31.131632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.132324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3ec011a6-6335-4572-8147-607024e03055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.133949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 989/1969] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:30 2022] DELETE /identity/v3/projects/a8dfc0cd97964af4a9c60c54a6ecc4b8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:31.140255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e833b-76cd-4eae-a441-24fe1ecd509b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.140739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e833b-76cd-4eae-a441-24fe1ecd509b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.141148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e833b-76cd-4eae-a441-24fe1ecd509b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.166639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d06e833b-76cd-4eae-a441-24fe1ecd509b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:31.167904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d06e833b-76cd-4eae-a441-24fe1ecd509b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:31.263929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 981/1970] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:31.277491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82482426-9fcd-4675-b049-a1973c249e72 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.278042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82482426-9fcd-4675-b049-a1973c249e72 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.278541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82482426-9fcd-4675-b049-a1973c249e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.321041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-82482426-9fcd-4675-b049-a1973c249e72 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:31.323301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-82482426-9fcd-4675-b049-a1973c249e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:31.337539 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 990/1971] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:31.342683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e833b-76cd-4eae-a441-24fe1ecd509b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:31.351610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:31.357230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e833b-76cd-4eae-a441-24fe1ecd509b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.363315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.363315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.364315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.364567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.365242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:31.365610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.367246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:31.385622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 982/1972] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:31.387252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82482426-9fcd-4675-b049-a1973c249e72 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:31.404823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82482426-9fcd-4675-b049-a1973c249e72 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.412068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5dd76f-f1ef-4d53-abea-903fc9b8010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.413132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dd76f-f1ef-4d53-abea-903fc9b8010d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.413690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dd76f-f1ef-4d53-abea-903fc9b8010d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.414171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dd76f-f1ef-4d53-abea-903fc9b8010d admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.415929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5dd76f-f1ef-4d53-abea-903fc9b8010d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.419921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5dd76f-f1ef-4d53-abea-903fc9b8010d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:31.461179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:31.463775 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:31.475964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 991/1973] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:31 2022] POST /identity/v3/projects => generated 362 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:31.481618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5dd76f-f1ef-4d53-abea-903fc9b8010d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:31.490821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d4ad3-3e1e-4875-a44d-76a46c7409e4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.570374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5dd76f-f1ef-4d53-abea-903fc9b8010d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.575318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bac78-d5c3-43fe-a69d-3b213c2f99cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.575930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bac78-d5c3-43fe-a69d-3b213c2f99cb admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.576212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bac78-d5c3-43fe-a69d-3b213c2f99cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.576500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bac78-d5c3-43fe-a69d-3b213c2f99cb admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.577114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bac78-d5c3-43fe-a69d-3b213c2f99cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.579077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bac78-d5c3-43fe-a69d-3b213c2f99cb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:31.596656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.598406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.599670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.599670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] PATH_INFO: `/v3/projects/7a48ae1d72bc4fbc9c7022961ac9b705` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.601131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a48ae1d72bc4fbc9c7022961ac9b705)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.605210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:31.608564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 992/1974] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:31 2022] POST /identity/v3/users => generated 282 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:31.609904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bac78-d5c3-43fe-a69d-3b213c2f99cb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:31.622121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bac78-d5c3-43fe-a69d-3b213c2f99cb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.634172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6371148-d46d-44de-aa3f-8ce6a604533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.634954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6371148-d46d-44de-aa3f-8ce6a604533c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.635237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6371148-d46d-44de-aa3f-8ce6a604533c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.635571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6371148-d46d-44de-aa3f-8ce6a604533c admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.636288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6371148-d46d-44de-aa3f-8ce6a604533c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.638984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6371148-d46d-44de-aa3f-8ce6a604533c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:31.652104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 993/1975] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:31 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:31.662217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6371148-d46d-44de-aa3f-8ce6a604533c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:31.664735 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:31.670070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6371148-d46d-44de-aa3f-8ce6a604533c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.678881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581050d-0952-4b57-a2de-e90c1fa88af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.679359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581050d-0952-4b57-a2de-e90c1fa88af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.679640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581050d-0952-4b57-a2de-e90c1fa88af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.679965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581050d-0952-4b57-a2de-e90c1fa88af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.680789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581050d-0952-4b57-a2de-e90c1fa88af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.682531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581050d-0952-4b57-a2de-e90c1fa88af9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:31.696091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 994/1976] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:31 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:31.700496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581050d-0952-4b57-a2de-e90c1fa88af9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:31.702813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a48ae1d72bc4fbc9c7022961ac9b705', 'request_id': 'req-8dbcb323-2be7-41c5-8785-2967c4d97e20'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.710863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581050d-0952-4b57-a2de-e90c1fa88af9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.713123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a48ae1d72bc4fbc9c7022961ac9b705', 'request_id': 'req-8dbcb323-2be7-41c5-8785-2967c4d97e20'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.722552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a48ae1d72bc4fbc9c7022961ac9b705', 'request_id': 'req-8dbcb323-2be7-41c5-8785-2967c4d97e20'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.745862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] The token cache is being invalidate because project 7a48ae1d72bc4fbc9c7022961ac9b705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:31.746723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.748242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:31.757748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 983/1977] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:31 2022] DELETE /identity/v3/projects/7a48ae1d72bc4fbc9c7022961ac9b705 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:31.759343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:31.761155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:31.809327 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb7daf-8076-46a8-bb88-ad98a582bbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.814254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb7daf-8076-46a8-bb88-ad98a582bbe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.815141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb7daf-8076-46a8-bb88-ad98a582bbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.815427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb7daf-8076-46a8-bb88-ad98a582bbe8 admin admin] PATH_INFO: `/v3/projects/5bb15284edad480e8207859530da4fa8/users/3dca7f7eb280450584313ce4a2b83168/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.843701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb7daf-8076-46a8-bb88-ad98a582bbe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb15284edad480e8207859530da4fa8, user_id=3dca7f7eb280450584313ce4a2b83168, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.847843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb7daf-8076-46a8-bb88-ad98a582bbe8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:31.870641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 995/1978] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:31 2022] PUT /identity/v3/projects/5bb15284edad480e8207859530da4fa8/users/3dca7f7eb280450584313ce4a2b83168/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:31.893399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcd248-534d-47a5-8194-34504d7f5b27 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.893399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcd248-534d-47a5-8194-34504d7f5b27 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.893399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcd248-534d-47a5-8194-34504d7f5b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.897322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcb323-2be7-41c5-8785-2967c4d97e20 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:31.928625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dafcd248-534d-47a5-8194-34504d7f5b27 None None] MFA Rules not processed for user `3dca7f7eb280450584313ce4a2b83168`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:31.929893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dafcd248-534d-47a5-8194-34504d7f5b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:31.950548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:31.988357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:31.988947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:31.989284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:31.989500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] PATH_INFO: `/v3/users/ad52c43ccdc442abbb551fceb3d017f7` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:31.990839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad52c43ccdc442abbb551fceb3d017f7)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:31.994100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:32.003332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 996/1979] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:32.047831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcd248-534d-47a5-8194-34504d7f5b27 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:32.049545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dafcd248-534d-47a5-8194-34504d7f5b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:32.068936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad52c43ccdc442abbb551fceb3d017f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.074892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad52c43ccdc442abbb551fceb3d017f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.076044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad52c43ccdc442abbb551fceb3d017f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.094415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad52c43ccdc442abbb551fceb3d017f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.111003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad52c43ccdc442abbb551fceb3d017f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.123307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 984/1980] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:31 2022] DELETE /identity/v3/users/ad52c43ccdc442abbb551fceb3d017f7 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:32.142487 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcd248-534d-47a5-8194-34504d7f5b27 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:32.164509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:32.200720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:32.201605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:32.201848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:32.202082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:32.204482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:32.271808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:32.273195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:32.298760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 997/1981] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:32.319981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:32.323416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:32.414345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12325d-458f-4415-9136-6b70c29008ef admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:32.426308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:32.427385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:32.428053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:32.428536 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] PATH_INFO: `/v3/users/3dca7f7eb280450584313ce4a2b83168` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:32.430033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dca7f7eb280450584313ce4a2b83168)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:32.432988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:32.522458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dca7f7eb280450584313ce4a2b83168'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.547868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:32.549712 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:32.549813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:32.556991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dca7f7eb280450584313ce4a2b83168'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.557873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dca7f7eb280450584313ce4a2b83168'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.587948 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dca7f7eb280450584313ce4a2b83168'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.605373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156abf3-4aff-41f3-a4c4-415bdb2c0079 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:32.614193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bb52658b-23aa-4b38-9ab8-66c21fb279be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dca7f7eb280450584313ce4a2b83168'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.621337 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:32.621961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:32.622250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:32.622568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] PATH_INFO: `/v3/projects/963c022af5584371bddd8690ce7ed097` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:32.624166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=963c022af5584371bddd8690ce7ed097)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:32.628517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 985/1982] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:32 2022] DELETE /identity/v3/users/3dca7f7eb280450584313ce4a2b83168 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:32.628844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:32.634190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfd29f-09ef-41aa-977e-b09765338454 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:32.634190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfd29f-09ef-41aa-977e-b09765338454 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:32.634698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfd29f-09ef-41aa-977e-b09765338454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:32.665003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-30dfd29f-09ef-41aa-977e-b09765338454 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:32.667006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-30dfd29f-09ef-41aa-977e-b09765338454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:32.710893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963c022af5584371bddd8690ce7ed097', 'request_id': 'req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.719551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963c022af5584371bddd8690ce7ed097', 'request_id': 'req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.725510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963c022af5584371bddd8690ce7ed097', 'request_id': 'req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.738917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] The token cache is being invalidate because project 963c022af5584371bddd8690ce7ed097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:32.739215 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.740515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5205ca7-1988-4074-8d0f-f6a58dfca0a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:32.742111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 998/1983] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:32 2022] DELETE /identity/v3/projects/963c022af5584371bddd8690ce7ed097 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:32.747282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c4065-fa2a-466c-ad38-81685af94d5c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:32.747670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c4065-fa2a-466c-ad38-81685af94d5c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:32.747943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c4065-fa2a-466c-ad38-81685af94d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:32.764678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 986/1984] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:32.770944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfd29f-09ef-41aa-977e-b09765338454 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:32.772458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-30dfd29f-09ef-41aa-977e-b09765338454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:32.775729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f26c4065-fa2a-466c-ad38-81685af94d5c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:32.777030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f26c4065-fa2a-466c-ad38-81685af94d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:32.852810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfd29f-09ef-41aa-977e-b09765338454 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:32.857615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:32.869743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 999/1985] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:32.883740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce599d01-c7c1-4d65-a6ba-2105a63bdc78 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:32.884785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce599d01-c7c1-4d65-a6ba-2105a63bdc78 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:32.885274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce599d01-c7c1-4d65-a6ba-2105a63bdc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:32.928064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c64ac2-11fd-450d-8ab3-864245020df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:32.928778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c64ac2-11fd-450d-8ab3-864245020df5 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:32.929210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c64ac2-11fd-450d-8ab3-864245020df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:32.929813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c64ac2-11fd-450d-8ab3-864245020df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:32.930363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ce599d01-c7c1-4d65-a6ba-2105a63bdc78 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:32.931969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ce599d01-c7c1-4d65-a6ba-2105a63bdc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:32.940449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c64ac2-11fd-450d-8ab3-864245020df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:32.941942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c64ac2-11fd-450d-8ab3-864245020df5 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:32.945276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1000/1986] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:32.948123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce599d01-c7c1-4d65-a6ba-2105a63bdc78 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:32.957969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 987/1987] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:32.966812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce599d01-c7c1-4d65-a6ba-2105a63bdc78 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:32.981425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b332ce-902d-4056-ba0a-80fd29fbe626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:32.981713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b332ce-902d-4056-ba0a-80fd29fbe626 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:32.981866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b332ce-902d-4056-ba0a-80fd29fbe626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:32.982184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b332ce-902d-4056-ba0a-80fd29fbe626 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:32.982873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b332ce-902d-4056-ba0a-80fd29fbe626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:32.983183 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b332ce-902d-4056-ba0a-80fd29fbe626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:32.984819 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b332ce-902d-4056-ba0a-80fd29fbe626 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:33.022886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1001/1988] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:33.027514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c64ac2-11fd-450d-8ab3-864245020df5 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:33.038429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c64ac2-11fd-450d-8ab3-864245020df5 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:33.043927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279eb033-2477-4483-a941-15b42079ff56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:33.044765 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279eb033-2477-4483-a941-15b42079ff56 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:33.044882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279eb033-2477-4483-a941-15b42079ff56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:33.045219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279eb033-2477-4483-a941-15b42079ff56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:33.045961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279eb033-2477-4483-a941-15b42079ff56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:33.048071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279eb033-2477-4483-a941-15b42079ff56 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:33.105945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 988/1989] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:33 2022] POST /identity/v3/projects => generated 380 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:33.108032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b332ce-902d-4056-ba0a-80fd29fbe626 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:33.165845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b332ce-902d-4056-ba0a-80fd29fbe626 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:33.297956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27458342-eb01-45bc-b427-aacd74acf395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:33.297956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27458342-eb01-45bc-b427-aacd74acf395 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:33.298441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27458342-eb01-45bc-b427-aacd74acf395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:33.298825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27458342-eb01-45bc-b427-aacd74acf395 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:33.299608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27458342-eb01-45bc-b427-aacd74acf395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:33.302002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27458342-eb01-45bc-b427-aacd74acf395 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:33.336416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1002/1990] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:33 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:33.341284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279eb033-2477-4483-a941-15b42079ff56 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:33.351088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:33.360441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279eb033-2477-4483-a941-15b42079ff56 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:33.375312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce2d4b-5256-4d29-8ba0-a94ce17aa886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:33.375943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce2d4b-5256-4d29-8ba0-a94ce17aa886 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:33.376226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce2d4b-5256-4d29-8ba0-a94ce17aa886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:33.376500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce2d4b-5256-4d29-8ba0-a94ce17aa886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:33.377160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce2d4b-5256-4d29-8ba0-a94ce17aa886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:33.378885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce2d4b-5256-4d29-8ba0-a94ce17aa886 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:33.400221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 989/1991] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:33 2022] GET /identity/v3/roles => generated 1427 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:33.402817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27458342-eb01-45bc-b427-aacd74acf395 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:33.427211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27458342-eb01-45bc-b427-aacd74acf395 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:33.437961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68383f-b284-4e55-b6b6-3bbe483bf422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:33.438969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68383f-b284-4e55-b6b6-3bbe483bf422 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:33.439383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68383f-b284-4e55-b6b6-3bbe483bf422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:33.439871 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68383f-b284-4e55-b6b6-3bbe483bf422 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:33.440955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68383f-b284-4e55-b6b6-3bbe483bf422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:33.444056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68383f-b284-4e55-b6b6-3bbe483bf422 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:33.476119 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1003/1992] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:33 2022] GET /identity/v3/roles => generated 1427 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:33.477126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce2d4b-5256-4d29-8ba0-a94ce17aa886 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:33.493093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce2d4b-5256-4d29-8ba0-a94ce17aa886 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:33.500317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fc9b4-10de-47c4-920e-f55da9ab43e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:33.501115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fc9b4-10de-47c4-920e-f55da9ab43e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:33.501464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fc9b4-10de-47c4-920e-f55da9ab43e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:33.502056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fc9b4-10de-47c4-920e-f55da9ab43e6 admin admin] PATH_INFO: `/v3/projects/96dcab83387040569f360f6dcef6fab9/users/0cf2fc2249fc44cd8c8bfbce934fa960/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:33.543880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fc9b4-10de-47c4-920e-f55da9ab43e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96dcab83387040569f360f6dcef6fab9, user_id=0cf2fc2249fc44cd8c8bfbce934fa960, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:33.548591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fc9b4-10de-47c4-920e-f55da9ab43e6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:33.571504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 990/1993] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:33 2022] PUT /identity/v3/projects/96dcab83387040569f360f6dcef6fab9/users/0cf2fc2249fc44cd8c8bfbce934fa960/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:33.582848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b5b78-b843-43b8-9173-b9be4c920f19 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:33.583434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b5b78-b843-43b8-9173-b9be4c920f19 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:33.584033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b5b78-b843-43b8-9173-b9be4c920f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:33.674152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-604b5b78-b843-43b8-9173-b9be4c920f19 None None] MFA Rules not processed for user `0cf2fc2249fc44cd8c8bfbce934fa960`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:33.674152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-604b5b78-b843-43b8-9173-b9be4c920f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:33.675966 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fc9b4-10de-47c4-920e-f55da9ab43e6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:33.686653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-850fc9b4-10de-47c4-920e-f55da9ab43e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:33.831964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1004/1994] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:33.840535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b5b78-b843-43b8-9173-b9be4c920f19 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:33.856171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b5b78-b843-43b8-9173-b9be4c920f19 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:33.861030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcd241-7efa-4862-87e5-27444326944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:33.861295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fc9b4-10de-47c4-920e-f55da9ab43e6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:33.861606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcd241-7efa-4862-87e5-27444326944b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:33.861773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcd241-7efa-4862-87e5-27444326944b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:33.862044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcd241-7efa-4862-87e5-27444326944b admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:33.862706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcd241-7efa-4862-87e5-27444326944b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:33.864619 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcd241-7efa-4862-87e5-27444326944b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:33.866400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:33.867225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:33.867772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:33.868049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] PATH_INFO: `/v3/projects/5bb15284edad480e8207859530da4fa8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:33.871029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb15284edad480e8207859530da4fa8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:33.873175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:33.899718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1005/1995] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:33 2022] POST /identity/v3/projects => generated 384 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:33.901271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcd241-7efa-4862-87e5-27444326944b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:33.934114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb15284edad480e8207859530da4fa8', 'request_id': 'req-70d9a1dd-26a9-4913-9885-16211e1d6d7c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:33.937705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:33.941250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb15284edad480e8207859530da4fa8', 'request_id': 'req-70d9a1dd-26a9-4913-9885-16211e1d6d7c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:33.950702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb15284edad480e8207859530da4fa8', 'request_id': 'req-70d9a1dd-26a9-4913-9885-16211e1d6d7c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:33.969196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] The token cache is being invalidate because project 5bb15284edad480e8207859530da4fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:33.969543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:33.970564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-70d9a1dd-26a9-4913-9885-16211e1d6d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:33.972497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 991/1996] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:33 2022] DELETE /identity/v3/projects/5bb15284edad480e8207859530da4fa8 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:33.977254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbe34-4c8b-4c2f-8301-5390905508f7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:33.977549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbe34-4c8b-4c2f-8301-5390905508f7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:33.977964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbe34-4c8b-4c2f-8301-5390905508f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:33.984507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcd241-7efa-4862-87e5-27444326944b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.001488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a3abbe34-4c8b-4c2f-8301-5390905508f7 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:34.003469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a3abbe34-4c8b-4c2f-8301-5390905508f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:34.054182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccdd2e-e8a4-44ed-aee7-e398097afa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.054696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccdd2e-e8a4-44ed-aee7-e398097afa7c admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.054976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccdd2e-e8a4-44ed-aee7-e398097afa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.055243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccdd2e-e8a4-44ed-aee7-e398097afa7c admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.056513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccdd2e-e8a4-44ed-aee7-e398097afa7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.058476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccdd2e-e8a4-44ed-aee7-e398097afa7c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.066042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:34.082032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 992/1997] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:34.083805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1006/1998] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:33 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:34.087981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abbe34-4c8b-4c2f-8301-5390905508f7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.089445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a3abbe34-4c8b-4c2f-8301-5390905508f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:34.091399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bef77d-852b-4d77-9352-3742be8bf793 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.091683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bef77d-852b-4d77-9352-3742be8bf793 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.091950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bef77d-852b-4d77-9352-3742be8bf793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.102584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abbe34-4c8b-4c2f-8301-5390905508f7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.109444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48cf0f-b3f4-48fb-a876-4a32af796e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.109444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48cf0f-b3f4-48fb-a876-4a32af796e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.109444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48cf0f-b3f4-48fb-a876-4a32af796e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.109444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48cf0f-b3f4-48fb-a876-4a32af796e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.109942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48cf0f-b3f4-48fb-a876-4a32af796e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.111179 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48cf0f-b3f4-48fb-a876-4a32af796e72 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.117076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-55bef77d-852b-4d77-9352-3742be8bf793 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:34.118970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-55bef77d-852b-4d77-9352-3742be8bf793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:34.123952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 993/1999] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:34 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:34.128303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48cf0f-b3f4-48fb-a876-4a32af796e72 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.129258 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1007/2000] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:34.133038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bef77d-852b-4d77-9352-3742be8bf793 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.140124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48cf0f-b3f4-48fb-a876-4a32af796e72 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.145229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bef77d-852b-4d77-9352-3742be8bf793 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.145846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baee24-22d1-48aa-af57-305fa89adcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.146351 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baee24-22d1-48aa-af57-305fa89adcca admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.146686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baee24-22d1-48aa-af57-305fa89adcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.146924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baee24-22d1-48aa-af57-305fa89adcca admin admin] PATH_INFO: `/v3/projects/97e026a1b5a7496ca57a3f4d9620faa7/users/1e374a0bb23641b4ae89cfbfb98b2895/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.150998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5496d-8b53-49e4-a213-381f25d741a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.154922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5496d-8b53-49e4-a213-381f25d741a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.155303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5496d-8b53-49e4-a213-381f25d741a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.155585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5496d-8b53-49e4-a213-381f25d741a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.156300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5496d-8b53-49e4-a213-381f25d741a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:34.156638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5496d-8b53-49e4-a213-381f25d741a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.158410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5496d-8b53-49e4-a213-381f25d741a1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.172072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1008/2001] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:37:34.178896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5496d-8b53-49e4-a213-381f25d741a1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.179501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baee24-22d1-48aa-af57-305fa89adcca admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e026a1b5a7496ca57a3f4d9620faa7, user_id=1e374a0bb23641b4ae89cfbfb98b2895, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.183320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baee24-22d1-48aa-af57-305fa89adcca admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.193253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5496d-8b53-49e4-a213-381f25d741a1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.206731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 994/2002] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:34 2022] PUT /identity/v3/projects/97e026a1b5a7496ca57a3f4d9620faa7/users/1e374a0bb23641b4ae89cfbfb98b2895/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:34.211497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baee24-22d1-48aa-af57-305fa89adcca admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.302304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6a764-004f-4c93-a67d-00931ef33d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.304054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6a764-004f-4c93-a67d-00931ef33d77 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.309879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6a764-004f-4c93-a67d-00931ef33d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.309879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6a764-004f-4c93-a67d-00931ef33d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.312018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6a764-004f-4c93-a67d-00931ef33d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.314559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6a764-004f-4c93-a67d-00931ef33d77 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.331339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baee24-22d1-48aa-af57-305fa89adcca admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.339069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4713f34-5fa7-49e1-a8c6-a78e734a5df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.341387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4713f34-5fa7-49e1-a8c6-a78e734a5df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.341968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4713f34-5fa7-49e1-a8c6-a78e734a5df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.342431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4713f34-5fa7-49e1-a8c6-a78e734a5df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.343454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4713f34-5fa7-49e1-a8c6-a78e734a5df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.345770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4713f34-5fa7-49e1-a8c6-a78e734a5df6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.356542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1009/2003] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:34 2022] POST /identity/v3/projects => generated 376 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:34.361477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6a764-004f-4c93-a67d-00931ef33d77 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.362254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 995/2004] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:34 2022] GET /identity/v3/roles => generated 1427 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:34.367843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4713f34-5fa7-49e1-a8c6-a78e734a5df6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.381929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4713f34-5fa7-49e1-a8c6-a78e734a5df6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.451388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:34.515157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f944a5-bbf4-45ec-be97-4cfde5210817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.515979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f944a5-bbf4-45ec-be97-4cfde5210817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.516511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f944a5-bbf4-45ec-be97-4cfde5210817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.516862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f944a5-bbf4-45ec-be97-4cfde5210817 admin admin] PATH_INFO: `/v3/projects/97e026a1b5a7496ca57a3f4d9620faa7/users/1e374a0bb23641b4ae89cfbfb98b2895/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.521675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f944a5-bbf4-45ec-be97-4cfde5210817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e026a1b5a7496ca57a3f4d9620faa7, user_id=1e374a0bb23641b4ae89cfbfb98b2895, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.524871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f944a5-bbf4-45ec-be97-4cfde5210817 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.530514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6a764-004f-4c93-a67d-00931ef33d77 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.542708 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 996/2005] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:34 2022] PUT /identity/v3/projects/97e026a1b5a7496ca57a3f4d9620faa7/users/1e374a0bb23641b4ae89cfbfb98b2895/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:34.553794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985129ce-6d4a-4dd8-84b4-fd9cdb573ab0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.554276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985129ce-6d4a-4dd8-84b4-fd9cdb573ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.554754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985129ce-6d4a-4dd8-84b4-fd9cdb573ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.601109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-985129ce-6d4a-4dd8-84b4-fd9cdb573ab0 None None] MFA Rules not processed for user `1e374a0bb23641b4ae89cfbfb98b2895`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:34.603572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-985129ce-6d4a-4dd8-84b4-fd9cdb573ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:34.630356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277af3f-4671-448c-b3c1-6da12a7725c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.631308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277af3f-4671-448c-b3c1-6da12a7725c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.631496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277af3f-4671-448c-b3c1-6da12a7725c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.631752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277af3f-4671-448c-b3c1-6da12a7725c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.632942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277af3f-4671-448c-b3c1-6da12a7725c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.636569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277af3f-4671-448c-b3c1-6da12a7725c1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.679560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1010/2006] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:34 2022] POST /identity/v3/users => generated 289 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:34.680913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277af3f-4671-448c-b3c1-6da12a7725c1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.699640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277af3f-4671-448c-b3c1-6da12a7725c1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.703907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905c0a7-c4e0-489d-88d8-55f80a3b3a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.704593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905c0a7-c4e0-489d-88d8-55f80a3b3a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.705154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905c0a7-c4e0-489d-88d8-55f80a3b3a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.705432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905c0a7-c4e0-489d-88d8-55f80a3b3a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.706385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905c0a7-c4e0-489d-88d8-55f80a3b3a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.706634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:34.708878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905c0a7-c4e0-489d-88d8-55f80a3b3a60 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.741340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1011/2007] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:34 2022] GET /identity/v3/roles => generated 1427 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:34.745477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905c0a7-c4e0-489d-88d8-55f80a3b3a60 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.758246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905c0a7-c4e0-489d-88d8-55f80a3b3a60 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.762143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 997/2008] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:34 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:34.767026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb978579-3cea-4f86-970e-689490d9059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.767920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb978579-3cea-4f86-970e-689490d9059a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.768242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb978579-3cea-4f86-970e-689490d9059a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.768620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb978579-3cea-4f86-970e-689490d9059a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.769395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb978579-3cea-4f86-970e-689490d9059a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.772208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb978579-3cea-4f86-970e-689490d9059a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.774987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985129ce-6d4a-4dd8-84b4-fd9cdb573ab0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.776798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-985129ce-6d4a-4dd8-84b4-fd9cdb573ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:34.784239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1012/2009] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:34 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:34.788786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb978579-3cea-4f86-970e-689490d9059a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:34.814000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb978579-3cea-4f86-970e-689490d9059a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.820543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd820b56-694a-4dbc-a24d-325afd68172b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.821338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd820b56-694a-4dbc-a24d-325afd68172b admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.821737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd820b56-694a-4dbc-a24d-325afd68172b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.822274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd820b56-694a-4dbc-a24d-325afd68172b admin admin] PATH_INFO: `/v3/projects/a930aab479e64135bdcdde16ea3b0a68/users/fc605668580b4de481fcc4e0f13694ae/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.864880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985129ce-6d4a-4dd8-84b4-fd9cdb573ab0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:34.870908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958580d6-2463-4e0c-a0c9-ce857925d5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:34.871649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958580d6-2463-4e0c-a0c9-ce857925d5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.871846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958580d6-2463-4e0c-a0c9-ce857925d5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.872159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958580d6-2463-4e0c-a0c9-ce857925d5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.872604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd820b56-694a-4dbc-a24d-325afd68172b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a930aab479e64135bdcdde16ea3b0a68, user_id=fc605668580b4de481fcc4e0f13694ae, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.883375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd820b56-694a-4dbc-a24d-325afd68172b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.890732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958580d6-2463-4e0c-a0c9-ce857925d5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:34.892871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958580d6-2463-4e0c-a0c9-ce857925d5a3 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:34.910729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 998/2010] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:34 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:34.912464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1013/2011] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:34 2022] PUT /identity/v3/projects/a930aab479e64135bdcdde16ea3b0a68/users/fc605668580b4de481fcc4e0f13694ae/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:34.931088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcafdf2-8fde-4117-be16-08badd846f94 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:34.931557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcafdf2-8fde-4117-be16-08badd846f94 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:34.932236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcafdf2-8fde-4117-be16-08badd846f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:34.991990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fbcafdf2-8fde-4117-be16-08badd846f94 None None] MFA Rules not processed for user `fc605668580b4de481fcc4e0f13694ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:34.994728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fbcafdf2-8fde-4117-be16-08badd846f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:35.069484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd820b56-694a-4dbc-a24d-325afd68172b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.127815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 999/2012] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:35.133313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcafdf2-8fde-4117-be16-08badd846f94 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.176297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd820b56-694a-4dbc-a24d-325afd68172b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:35.180966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:35.181916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.182197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.182471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.184922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:35.244207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:35.248199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcafdf2-8fde-4117-be16-08badd846f94 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:35.254032 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5827aec-113d-4ecf-bf40-fac318d13cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:35.254737 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5827aec-113d-4ecf-bf40-fac318d13cdc admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.255084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5827aec-113d-4ecf-bf40-fac318d13cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.255652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5827aec-113d-4ecf-bf40-fac318d13cdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.256661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5827aec-113d-4ecf-bf40-fac318d13cdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:35.259676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5827aec-113d-4ecf-bf40-fac318d13cdc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:35.286309 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:35.287223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:35.288491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:35.295888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1000/2013] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:35 2022] POST /identity/v3/projects => generated 376 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:35.299784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5827aec-113d-4ecf-bf40-fac318d13cdc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.302954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1014/2014] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:35.392559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5827aec-113d-4ecf-bf40-fac318d13cdc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:35.398038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c1de5-d3e6-4878-8a13-d7f341833f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:35.398687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c1de5-d3e6-4878-8a13-d7f341833f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.398968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c1de5-d3e6-4878-8a13-d7f341833f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.399235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c1de5-d3e6-4878-8a13-d7f341833f99 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.402539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c1de5-d3e6-4878-8a13-d7f341833f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:35.404031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.404924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c1de5-d3e6-4878-8a13-d7f341833f99 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:35.405768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:35.423978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eae849-0eed-4739-9dec-82dd50552754 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:35.433488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:35.434658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.435112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.435547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] PATH_INFO: `/v3/users/341fc92dd6ef49d6b18ef1f0f70543f2` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.437076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=341fc92dd6ef49d6b18ef1f0f70543f2)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:35.439701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:35.440576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1001/2015] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:35 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:35.442992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c1de5-d3e6-4878-8a13-d7f341833f99 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.457095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c1de5-d3e6-4878-8a13-d7f341833f99 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:35.462541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed4381-6e8b-44e2-852f-83676ac08039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:35.464117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed4381-6e8b-44e2-852f-83676ac08039 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.464403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed4381-6e8b-44e2-852f-83676ac08039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.464681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed4381-6e8b-44e2-852f-83676ac08039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.465637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed4381-6e8b-44e2-852f-83676ac08039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:35.467420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed4381-6e8b-44e2-852f-83676ac08039 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:35.480602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1002/2016] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:35 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:35.484504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed4381-6e8b-44e2-852f-83676ac08039 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.499200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed4381-6e8b-44e2-852f-83676ac08039 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:35.512775 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e56630-29b2-448b-9e11-80c97bcebc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:35.513628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e56630-29b2-448b-9e11-80c97bcebc14 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.514220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e56630-29b2-448b-9e11-80c97bcebc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.514609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e56630-29b2-448b-9e11-80c97bcebc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.515632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e56630-29b2-448b-9e11-80c97bcebc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:35.517303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e56630-29b2-448b-9e11-80c97bcebc14 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:35.521953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '341fc92dd6ef49d6b18ef1f0f70543f2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.531523 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1003/2017] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:35 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:35.537193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e56630-29b2-448b-9e11-80c97bcebc14 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.544069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '341fc92dd6ef49d6b18ef1f0f70543f2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.551594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e56630-29b2-448b-9e11-80c97bcebc14 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:35.553530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '341fc92dd6ef49d6b18ef1f0f70543f2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.557501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aea9e-8107-4434-aea2-a4cdb619b666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:35.558215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aea9e-8107-4434-aea2-a4cdb619b666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.558493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aea9e-8107-4434-aea2-a4cdb619b666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.558827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aea9e-8107-4434-aea2-a4cdb619b666 admin admin] PATH_INFO: `/v3/projects/3c4cf944302f45b18e108e8767a28368/users/7b6dea47df884eb4ab7885e0622f670d/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.564902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '341fc92dd6ef49d6b18ef1f0f70543f2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.565933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1b33e68d-5382-4691-87a9-f90489e0bc11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '341fc92dd6ef49d6b18ef1f0f70543f2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.581393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1aea9e-8107-4434-aea2-a4cdb619b666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4cf944302f45b18e108e8767a28368, user_id=7b6dea47df884eb4ab7885e0622f670d, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:35.583255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1015/2018] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:35 2022] DELETE /identity/v3/users/341fc92dd6ef49d6b18ef1f0f70543f2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:35.585166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1aea9e-8107-4434-aea2-a4cdb619b666 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:35.589280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4a690-a4fd-4874-aacd-0c7b3fd47170 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.589597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4a690-a4fd-4874-aacd-0c7b3fd47170 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.589960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4a690-a4fd-4874-aacd-0c7b3fd47170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.612684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1004/2019] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:35 2022] PUT /identity/v3/projects/3c4cf944302f45b18e108e8767a28368/users/7b6dea47df884eb4ab7885e0622f670d/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:35.625638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bcd53-9a10-41f5-8bf6-7aae7ed5a30e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.626303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bcd53-9a10-41f5-8bf6-7aae7ed5a30e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.626777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bcd53-9a10-41f5-8bf6-7aae7ed5a30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.626947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b1f4a690-a4fd-4874-aacd-0c7b3fd47170 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:35.628188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b1f4a690-a4fd-4874-aacd-0c7b3fd47170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:35.676884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-173bcd53-9a10-41f5-8bf6-7aae7ed5a30e None None] MFA Rules not processed for user `7b6dea47df884eb4ab7885e0622f670d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:35.678190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-173bcd53-9a10-41f5-8bf6-7aae7ed5a30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:35.729247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1016/2020] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:35.733364 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4a690-a4fd-4874-aacd-0c7b3fd47170 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.736173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b1f4a690-a4fd-4874-aacd-0c7b3fd47170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:35.753031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4a690-a4fd-4874-aacd-0c7b3fd47170 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:35.758604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:35.760084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.760460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.760941 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] PATH_INFO: `/v3/users/0cf2fc2249fc44cd8c8bfbce934fa960` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.762629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf2fc2249fc44cd8c8bfbce934fa960)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:35.764734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:35.777693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1005/2021] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:35.782284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bcd53-9a10-41f5-8bf6-7aae7ed5a30e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.784199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-173bcd53-9a10-41f5-8bf6-7aae7ed5a30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:35.828301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:35.843235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:35.853858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf2fc2249fc44cd8c8bfbce934fa960'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.872207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf2fc2249fc44cd8c8bfbce934fa960'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.881624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bcd53-9a10-41f5-8bf6-7aae7ed5a30e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:35.887593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbab000-d0b3-4e13-bc11-d02dc1e7e87c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:35.888252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbab000-d0b3-4e13-bc11-d02dc1e7e87c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.888494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbab000-d0b3-4e13-bc11-d02dc1e7e87c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.888828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbab000-d0b3-4e13-bc11-d02dc1e7e87c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.889642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf2fc2249fc44cd8c8bfbce934fa960'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.900686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbab000-d0b3-4e13-bc11-d02dc1e7e87c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:35.900968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf2fc2249fc44cd8c8bfbce934fa960'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.901617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf2fc2249fc44cd8c8bfbce934fa960'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:35.902500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbab000-d0b3-4e13-bc11-d02dc1e7e87c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:35.909307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1017/2022] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:35 2022] DELETE /identity/v3/users/0cf2fc2249fc44cd8c8bfbce934fa960 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:35.910204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:35.914097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:35.914334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1006/2023] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:35.916337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55bb30-81c5-42ec-b6b4-f50b2d194ff1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:35.916796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55bb30-81c5-42ec-b6b4-f50b2d194ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:35.917136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55bb30-81c5-42ec-b6b4-f50b2d194ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:35.949513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-da55bb30-81c5-42ec-b6b4-f50b2d194ff1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:35.951319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-da55bb30-81c5-42ec-b6b4-f50b2d194ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:36.013124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b293b3-6bbc-4d3d-9828-08d79d8aadbf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.018869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f8b75-c33f-4482-9835-f28ae300f02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.019905 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f8b75-c33f-4482-9835-f28ae300f02d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.020375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f8b75-c33f-4482-9835-f28ae300f02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.020892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f8b75-c33f-4482-9835-f28ae300f02d admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.022008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f8b75-c33f-4482-9835-f28ae300f02d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.025158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f8b75-c33f-4482-9835-f28ae300f02d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.060213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1018/2024] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:35 2022] POST /identity/v3/projects => generated 376 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:36.061477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f8b75-c33f-4482-9835-f28ae300f02d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.061971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1007/2025] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:36.073618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55bb30-81c5-42ec-b6b4-f50b2d194ff1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.075605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-da55bb30-81c5-42ec-b6b4-f50b2d194ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:36.145443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f8b75-c33f-4482-9835-f28ae300f02d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.151369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79904ef-47c6-41fc-9003-2b22b2be5a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.151998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79904ef-47c6-41fc-9003-2b22b2be5a20 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.152279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79904ef-47c6-41fc-9003-2b22b2be5a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.152551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79904ef-47c6-41fc-9003-2b22b2be5a20 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.153189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79904ef-47c6-41fc-9003-2b22b2be5a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.155177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79904ef-47c6-41fc-9003-2b22b2be5a20 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.164564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55bb30-81c5-42ec-b6b4-f50b2d194ff1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.187190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1019/2026] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:36 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:36.191723 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79904ef-47c6-41fc-9003-2b22b2be5a20 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.204534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79904ef-47c6-41fc-9003-2b22b2be5a20 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.211501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e87818-b87b-4875-8168-5f9f837fe67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.212218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e87818-b87b-4875-8168-5f9f837fe67a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.212497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e87818-b87b-4875-8168-5f9f837fe67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.213175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e87818-b87b-4875-8168-5f9f837fe67a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.214015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e87818-b87b-4875-8168-5f9f837fe67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.215652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e87818-b87b-4875-8168-5f9f837fe67a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.235273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1020/2027] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:36 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:36.236512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e87818-b87b-4875-8168-5f9f837fe67a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.239061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962befb-f54c-45b4-b06f-b91ddf5cf79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.239877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962befb-f54c-45b4-b06f-b91ddf5cf79a service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.240331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962befb-f54c-45b4-b06f-b91ddf5cf79a service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.240859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962befb-f54c-45b4-b06f-b91ddf5cf79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.249926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e87818-b87b-4875-8168-5f9f837fe67a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.256105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f988e-74eb-4b18-934d-43466fe0ae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.256798 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f988e-74eb-4b18-934d-43466fe0ae8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.257082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f988e-74eb-4b18-934d-43466fe0ae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.257391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f988e-74eb-4b18-934d-43466fe0ae8f admin admin] PATH_INFO: `/v3/projects/d0db4f8425f34f9792a2638042639695/users/dbc8c7d5f213404fab22df19021b3558/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.260739 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962befb-f54c-45b4-b06f-b91ddf5cf79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.264355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962befb-f54c-45b4-b06f-b91ddf5cf79a service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.285002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f988e-74eb-4b18-934d-43466fe0ae8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0db4f8425f34f9792a2638042639695, user_id=dbc8c7d5f213404fab22df19021b3558, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.286819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1008/2028] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:36.292951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f988e-74eb-4b18-934d-43466fe0ae8f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.304421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:36.316796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962befb-f54c-45b4-b06f-b91ddf5cf79a service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.319428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1021/2029] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:36 2022] PUT /identity/v3/projects/d0db4f8425f34f9792a2638042639695/users/dbc8c7d5f213404fab22df19021b3558/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:36.320298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3962befb-f54c-45b4-b06f-b91ddf5cf79a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:36.327328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f988e-74eb-4b18-934d-43466fe0ae8f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.335539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962befb-f54c-45b4-b06f-b91ddf5cf79a service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.435534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f988e-74eb-4b18-934d-43466fe0ae8f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.444006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:36.448195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4a866-c220-412c-a941-f36016d0a567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.448838 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4a866-c220-412c-a941-f36016d0a567 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.449172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4a866-c220-412c-a941-f36016d0a567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.449455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4a866-c220-412c-a941-f36016d0a567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.450831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4a866-c220-412c-a941-f36016d0a567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.452605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4a866-c220-412c-a941-f36016d0a567 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.468057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1022/2030] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:36 2022] GET /identity/v3/roles => generated 1427 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:36.470097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4a866-c220-412c-a941-f36016d0a567 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.484571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4a866-c220-412c-a941-f36016d0a567 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.485043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.485901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.486322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.486598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] PATH_INFO: `/v3/projects/843dd8b5651d4a9096effa8b264b6ace` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.488020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=843dd8b5651d4a9096effa8b264b6ace)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.490382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.500333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f6eea-c6f8-405a-bf32-ce4ed605ffaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.502852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f6eea-c6f8-405a-bf32-ce4ed605ffaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.503214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f6eea-c6f8-405a-bf32-ce4ed605ffaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.503458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f6eea-c6f8-405a-bf32-ce4ed605ffaa admin admin] PATH_INFO: `/v3/projects/d0db4f8425f34f9792a2638042639695/users/dbc8c7d5f213404fab22df19021b3558/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.505686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f6eea-c6f8-405a-bf32-ce4ed605ffaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0db4f8425f34f9792a2638042639695, user_id=dbc8c7d5f213404fab22df19021b3558, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.509806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f6eea-c6f8-405a-bf32-ce4ed605ffaa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.526156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1023/2031] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:36 2022] PUT /identity/v3/projects/d0db4f8425f34f9792a2638042639695/users/dbc8c7d5f213404fab22df19021b3558/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:36.544251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7c78a-6e16-4537-b2d3-44697e891b5b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.544547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7c78a-6e16-4537-b2d3-44697e891b5b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.544840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7c78a-6e16-4537-b2d3-44697e891b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.564457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '843dd8b5651d4a9096effa8b264b6ace', 'request_id': 'req-52583738-5195-4558-afea-dafd0d1ee238'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.571548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '843dd8b5651d4a9096effa8b264b6ace', 'request_id': 'req-52583738-5195-4558-afea-dafd0d1ee238'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.579423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '843dd8b5651d4a9096effa8b264b6ace', 'request_id': 'req-52583738-5195-4558-afea-dafd0d1ee238'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.581468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c5d7c78a-6e16-4537-b2d3-44697e891b5b None None] MFA Rules not processed for user `dbc8c7d5f213404fab22df19021b3558`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:36.582738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c5d7c78a-6e16-4537-b2d3-44697e891b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:36.600217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] The token cache is being invalidate because project 843dd8b5651d4a9096effa8b264b6ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:36.600600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.601355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.604162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1009/2032] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:36 2022] DELETE /identity/v3/projects/843dd8b5651d4a9096effa8b264b6ace => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:36.608259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.609969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:36.695271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52583738-5195-4558-afea-dafd0d1ee238 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.700948 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.701667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.701942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.702217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] PATH_INFO: `/v3/users/d0a5920e58894979837e43d1974873be` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.703895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a5920e58894979837e43d1974873be)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.705852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.708632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1024/2033] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:36.722095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7c78a-6e16-4537-b2d3-44697e891b5b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.723787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c5d7c78a-6e16-4537-b2d3-44697e891b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:36.817591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a5920e58894979837e43d1974873be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.828526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a5920e58894979837e43d1974873be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.829921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a5920e58894979837e43d1974873be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.840946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:36.843879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7c78a-6e16-4537-b2d3-44697e891b5b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:36.851445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:36.852075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:36.852361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:36.852711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:36.853939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a5920e58894979837e43d1974873be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.854577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:36.864400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a5920e58894979837e43d1974873be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:36.873593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1010/2034] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:36 2022] DELETE /identity/v3/users/d0a5920e58894979837e43d1974873be => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:36.874611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:36.876211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:36.897179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:36.974984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:36.977171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:36.990140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1025/2035] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:37.025067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff32ad-95f2-4de9-a102-44693b442e18 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:37.031916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:37.032744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.033033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.033362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] PATH_INFO: `/v3/projects/96dcab83387040569f360f6dcef6fab9` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.034729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96dcab83387040569f360f6dcef6fab9)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:37.038720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:37.109997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96dcab83387040569f360f6dcef6fab9', 'request_id': 'req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.119437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96dcab83387040569f360f6dcef6fab9', 'request_id': 'req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.127453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96dcab83387040569f360f6dcef6fab9', 'request_id': 'req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.160641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] The token cache is being invalidate because project 96dcab83387040569f360f6dcef6fab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:37.161009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.161935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.164255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1011/2036] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:36 2022] DELETE /identity/v3/projects/96dcab83387040569f360f6dcef6fab9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:37.168824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:37.170497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:37.185481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4c0d1-c627-4380-9e97-c566887355b6 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:37.270337 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:37.271084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.271263 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.271715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] PATH_INFO: `/v3/users/1e374a0bb23641b4ae89cfbfb98b2895` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.273416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e374a0bb23641b4ae89cfbfb98b2895)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:37.275839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:37.311091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:37.312975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:37.329656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4d3e5-a2fd-4ac9-8b0b-73d01c0c7296 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:37.334497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:37.335157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.335498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.335774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] PATH_INFO: `/v3/projects/5602ec31052e4716845e3df6324a8d7a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.337057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5602ec31052e4716845e3df6324a8d7a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:37.339118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:37.397757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e374a0bb23641b4ae89cfbfb98b2895'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.418560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5602ec31052e4716845e3df6324a8d7a', 'request_id': 'req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.427000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5602ec31052e4716845e3df6324a8d7a', 'request_id': 'req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.431935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e374a0bb23641b4ae89cfbfb98b2895'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.434736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5602ec31052e4716845e3df6324a8d7a', 'request_id': 'req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.439741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:37.454113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e374a0bb23641b4ae89cfbfb98b2895'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.459524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] The token cache is being invalidate because project 5602ec31052e4716845e3df6324a8d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:37.460006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.462791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-96b6ca84-39ea-482e-9b1d-96dd1a15b1bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.465367 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1012/2037] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:37 2022] DELETE /identity/v3/projects/5602ec31052e4716845e3df6324a8d7a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:37.471938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc5b28-9a25-4384-87fa-2b12c79526b6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.472322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc5b28-9a25-4384-87fa-2b12c79526b6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.472606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc5b28-9a25-4384-87fa-2b12c79526b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.474726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e374a0bb23641b4ae89cfbfb98b2895'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.476251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dcf15bc0-5387-4082-be1c-2f3e87b5d216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e374a0bb23641b4ae89cfbfb98b2895'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.497598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d7fc5b28-9a25-4384-87fa-2b12c79526b6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:37.502332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1026/2038] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:37 2022] DELETE /identity/v3/users/1e374a0bb23641b4ae89cfbfb98b2895 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:37.502508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d7fc5b28-9a25-4384-87fa-2b12c79526b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:37.604987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1013/2039] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:37.614615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08155c-33ee-4596-adb6-28b4098048ff None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.615156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08155c-33ee-4596-adb6-28b4098048ff None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.615652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08155c-33ee-4596-adb6-28b4098048ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.654638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4a08155c-33ee-4596-adb6-28b4098048ff None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:37.656200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4a08155c-33ee-4596-adb6-28b4098048ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:37.667700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1027/2040] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:37.671881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc5b28-9a25-4384-87fa-2b12c79526b6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:37.683810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc5b28-9a25-4384-87fa-2b12c79526b6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:37.689373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7838d91-24d3-4a0a-8af5-7196cdddec43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:37.690046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7838d91-24d3-4a0a-8af5-7196cdddec43 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.690460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7838d91-24d3-4a0a-8af5-7196cdddec43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.690782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7838d91-24d3-4a0a-8af5-7196cdddec43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.691597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7838d91-24d3-4a0a-8af5-7196cdddec43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:37.691954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7838d91-24d3-4a0a-8af5-7196cdddec43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:37.693721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7838d91-24d3-4a0a-8af5-7196cdddec43 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:37.710342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1014/2041] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:37.714305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08155c-33ee-4596-adb6-28b4098048ff None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:37.730361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08155c-33ee-4596-adb6-28b4098048ff None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:37.735129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35c018-81cc-41c2-81dd-4696bf2629d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:37.735741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35c018-81cc-41c2-81dd-4696bf2629d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.736021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35c018-81cc-41c2-81dd-4696bf2629d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.736293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35c018-81cc-41c2-81dd-4696bf2629d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.737485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35c018-81cc-41c2-81dd-4696bf2629d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:37.738932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35c018-81cc-41c2-81dd-4696bf2629d0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:37.771541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1028/2042] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:37 2022] POST /identity/v3/projects => generated 374 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:37.772688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7838d91-24d3-4a0a-8af5-7196cdddec43 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:37.783586 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7838d91-24d3-4a0a-8af5-7196cdddec43 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:37.845962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb330d87-bfa7-4710-ad78-cc03eb623564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:37.846754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb330d87-bfa7-4710-ad78-cc03eb623564 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.847093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb330d87-bfa7-4710-ad78-cc03eb623564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.847373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb330d87-bfa7-4710-ad78-cc03eb623564 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.848070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb330d87-bfa7-4710-ad78-cc03eb623564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:37.850152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb330d87-bfa7-4710-ad78-cc03eb623564 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:37.856820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35c018-81cc-41c2-81dd-4696bf2629d0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:37.858793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ae35c018-81cc-41c2-81dd-4696bf2629d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:37.875415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35c018-81cc-41c2-81dd-4696bf2629d0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:37.876571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1015/2043] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:37 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:37.880424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:37.880744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb330d87-bfa7-4710-ad78-cc03eb623564 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:37.881296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.881609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.881883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] PATH_INFO: `/v3/projects/97e026a1b5a7496ca57a3f4d9620faa7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.883018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97e026a1b5a7496ca57a3f4d9620faa7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:37.886140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:37.891529 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb330d87-bfa7-4710-ad78-cc03eb623564 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:37.893480 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:37.896707 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90d601-13c8-4cd2-8504-e631b8872b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:37.897313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90d601-13c8-4cd2-8504-e631b8872b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.897607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90d601-13c8-4cd2-8504-e631b8872b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.897953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90d601-13c8-4cd2-8504-e631b8872b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.898730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90d601-13c8-4cd2-8504-e631b8872b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:37.900882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90d601-13c8-4cd2-8504-e631b8872b0b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:37.913971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1016/2044] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:37 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:37.918548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90d601-13c8-4cd2-8504-e631b8872b0b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:37.930046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90d601-13c8-4cd2-8504-e631b8872b0b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:37.940808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97e026a1b5a7496ca57a3f4d9620faa7', 'request_id': 'req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.949291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97e026a1b5a7496ca57a3f4d9620faa7', 'request_id': 'req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.955336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97e026a1b5a7496ca57a3f4d9620faa7', 'request_id': 'req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.969874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] The token cache is being invalidate because project 97e026a1b5a7496ca57a3f4d9620faa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:37.970172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.970858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-183b9e0d-a1b0-4a18-af2b-f7341dbe0748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:37.972424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1029/2045] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:37 2022] DELETE /identity/v3/projects/97e026a1b5a7496ca57a3f4d9620faa7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:37.977289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3a6a4-9a0a-4c4f-8bb1-319ad209b42b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.978515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3a6a4-9a0a-4c4f-8bb1-319ad209b42b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.979080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3a6a4-9a0a-4c4f-8bb1-319ad209b42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.997594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63da65-3202-40c1-8004-452be9fd601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:37.998230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63da65-3202-40c1-8004-452be9fd601c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:37.998512 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63da65-3202-40c1-8004-452be9fd601c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:37.998786 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63da65-3202-40c1-8004-452be9fd601c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:37.999430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63da65-3202-40c1-8004-452be9fd601c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.001053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63da65-3202-40c1-8004-452be9fd601c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.003721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:38.008994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-34a3a6a4-9a0a-4c4f-8bb1-319ad209b42b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:38.010716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34a3a6a4-9a0a-4c4f-8bb1-319ad209b42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:38.013503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1017/2046] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:37 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:38.017697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63da65-3202-40c1-8004-452be9fd601c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.019721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7f63da65-3202-40c1-8004-452be9fd601c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:38.025506 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1030/2047] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:38.033103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d805f6-3441-48e8-aef4-7349ffafe3ba None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.033388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d805f6-3441-48e8-aef4-7349ffafe3ba None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.033694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d805f6-3441-48e8-aef4-7349ffafe3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.033828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63da65-3202-40c1-8004-452be9fd601c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.042132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc991cc9-021e-422b-9d73-ef362ac61734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.042842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc991cc9-021e-422b-9d73-ef362ac61734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.043126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc991cc9-021e-422b-9d73-ef362ac61734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.043403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc991cc9-021e-422b-9d73-ef362ac61734 admin admin] PATH_INFO: `/v3/projects/ca305d7effb241579d6420284bc66d4e/users/cfb2b8eccc84420ab59b9ff9d8582d26/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.057162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-22d805f6-3441-48e8-aef4-7349ffafe3ba None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:38.058435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-22d805f6-3441-48e8-aef4-7349ffafe3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:38.061101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc991cc9-021e-422b-9d73-ef362ac61734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca305d7effb241579d6420284bc66d4e, user_id=cfb2b8eccc84420ab59b9ff9d8582d26, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.064022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc991cc9-021e-422b-9d73-ef362ac61734 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.067616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1031/2048] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:38.072513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d805f6-3441-48e8-aef4-7349ffafe3ba None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.080356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1018/2049] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:38 2022] PUT /identity/v3/projects/ca305d7effb241579d6420284bc66d4e/users/cfb2b8eccc84420ab59b9ff9d8582d26/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:38.082032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d805f6-3441-48e8-aef4-7349ffafe3ba None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.091660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d2a1-115d-4b1c-a7fe-2fc6c4fefca6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.091935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d2a1-115d-4b1c-a7fe-2fc6c4fefca6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.092216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d2a1-115d-4b1c-a7fe-2fc6c4fefca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.124273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e9c3d2a1-115d-4b1c-a7fe-2fc6c4fefca6 None None] MFA Rules not processed for user `cfb2b8eccc84420ab59b9ff9d8582d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:38.125689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e9c3d2a1-115d-4b1c-a7fe-2fc6c4fefca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:38.170449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148f29a-19cf-4e3e-a28f-cd25f0b14a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.171066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148f29a-19cf-4e3e-a28f-cd25f0b14a79 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.171230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148f29a-19cf-4e3e-a28f-cd25f0b14a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.171499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148f29a-19cf-4e3e-a28f-cd25f0b14a79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.173018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148f29a-19cf-4e3e-a28f-cd25f0b14a79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:38.173018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148f29a-19cf-4e3e-a28f-cd25f0b14a79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.174140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148f29a-19cf-4e3e-a28f-cd25f0b14a79 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.189445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1032/2050] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:37:38.194972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148f29a-19cf-4e3e-a28f-cd25f0b14a79 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.200407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1019/2051] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:38.205221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3d2a1-115d-4b1c-a7fe-2fc6c4fefca6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.208294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148f29a-19cf-4e3e-a28f-cd25f0b14a79 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.214234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622488-80d3-4333-a306-0d3ca0050504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.214856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622488-80d3-4333-a306-0d3ca0050504 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.215149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622488-80d3-4333-a306-0d3ca0050504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.215422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622488-80d3-4333-a306-0d3ca0050504 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.216096 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622488-80d3-4333-a306-0d3ca0050504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.218063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622488-80d3-4333-a306-0d3ca0050504 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.218450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3d2a1-115d-4b1c-a7fe-2fc6c4fefca6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.223271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb97c0f-e352-4aec-9af4-3a06493dc043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.223916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb97c0f-e352-4aec-9af4-3a06493dc043 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.224195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb97c0f-e352-4aec-9af4-3a06493dc043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.224459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb97c0f-e352-4aec-9af4-3a06493dc043 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.225096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb97c0f-e352-4aec-9af4-3a06493dc043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.227280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb97c0f-e352-4aec-9af4-3a06493dc043 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.250252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1033/2052] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:38 2022] POST /identity/v3/projects => generated 384 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:38.255423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622488-80d3-4333-a306-0d3ca0050504 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.266821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1020/2053] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:38 2022] POST /identity/v3/projects => generated 374 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:38.268101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb97c0f-e352-4aec-9af4-3a06493dc043 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.345907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:38.360117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622488-80d3-4333-a306-0d3ca0050504 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.382043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb97c0f-e352-4aec-9af4-3a06493dc043 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.389457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55f40d-ae39-400f-add8-a9067210d7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.390617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55f40d-ae39-400f-add8-a9067210d7ef admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.391186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55f40d-ae39-400f-add8-a9067210d7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.391645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55f40d-ae39-400f-add8-a9067210d7ef admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.393011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55f40d-ae39-400f-add8-a9067210d7ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.395711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55f40d-ae39-400f-add8-a9067210d7ef admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.426267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1021/2054] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:38 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:38.429930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55f40d-ae39-400f-add8-a9067210d7ef admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.464176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55f40d-ae39-400f-add8-a9067210d7ef admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.478994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dd012-da3e-41ab-9e09-930b3edf18f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.479649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dd012-da3e-41ab-9e09-930b3edf18f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.484291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bea7f-e9e6-4512-9910-bb32e3cafd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.484503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dd012-da3e-41ab-9e09-930b3edf18f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.484503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dd012-da3e-41ab-9e09-930b3edf18f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.484665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bea7f-e9e6-4512-9910-bb32e3cafd8c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.484665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bea7f-e9e6-4512-9910-bb32e3cafd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.484801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457dd012-da3e-41ab-9e09-930b3edf18f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.484869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bea7f-e9e6-4512-9910-bb32e3cafd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.484869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bea7f-e9e6-4512-9910-bb32e3cafd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.485009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457dd012-da3e-41ab-9e09-930b3edf18f5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.485080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bea7f-e9e6-4512-9910-bb32e3cafd8c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.507887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1022/2055] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:38 2022] GET /identity/v3/roles => generated 1427 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:38.512926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bea7f-e9e6-4512-9910-bb32e3cafd8c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.525439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bea7f-e9e6-4512-9910-bb32e3cafd8c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.534124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c2d8f-abfd-47b5-ac4f-60361adfca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.534840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c2d8f-abfd-47b5-ac4f-60361adfca8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.535693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c2d8f-abfd-47b5-ac4f-60361adfca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.535982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c2d8f-abfd-47b5-ac4f-60361adfca8e admin admin] PATH_INFO: `/v3/projects/621e741a67af464a9d16641a24c13bdc/users/939a3320873f4a1db47ea0b23da427ea/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.556194 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c2d8f-abfd-47b5-ac4f-60361adfca8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=621e741a67af464a9d16641a24c13bdc, user_id=939a3320873f4a1db47ea0b23da427ea, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.559042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c2d8f-abfd-47b5-ac4f-60361adfca8e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.563146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:38.564235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1034/2056] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:38 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:38.574624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dd012-da3e-41ab-9e09-930b3edf18f5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.579865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1023/2057] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:38 2022] PUT /identity/v3/projects/621e741a67af464a9d16641a24c13bdc/users/939a3320873f4a1db47ea0b23da427ea/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:38.582671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c2d8f-abfd-47b5-ac4f-60361adfca8e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.590231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dd012-da3e-41ab-9e09-930b3edf18f5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.657675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c2d8f-abfd-47b5-ac4f-60361adfca8e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.664028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5b82f-10b7-4f55-b8ef-a47e7913349a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.664648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5b82f-10b7-4f55-b8ef-a47e7913349a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.665850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5b82f-10b7-4f55-b8ef-a47e7913349a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.666180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5b82f-10b7-4f55-b8ef-a47e7913349a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.666854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5b82f-10b7-4f55-b8ef-a47e7913349a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.667292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49316e2b-14dd-4599-bcb9-a5cb79773afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.668027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49316e2b-14dd-4599-bcb9-a5cb79773afb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.668380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49316e2b-14dd-4599-bcb9-a5cb79773afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.668497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5b82f-10b7-4f55-b8ef-a47e7913349a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.668672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49316e2b-14dd-4599-bcb9-a5cb79773afb admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.669426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49316e2b-14dd-4599-bcb9-a5cb79773afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.671388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49316e2b-14dd-4599-bcb9-a5cb79773afb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.681702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1024/2058] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:38 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:38.686427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5b82f-10b7-4f55-b8ef-a47e7913349a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.697516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1035/2059] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:38 2022] GET /identity/v3/roles => generated 1427 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:38.699035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49316e2b-14dd-4599-bcb9-a5cb79773afb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.699802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5b82f-10b7-4f55-b8ef-a47e7913349a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.704663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67065c-a767-4b31-a064-8b5811a9e185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.705592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67065c-a767-4b31-a064-8b5811a9e185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.705880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67065c-a767-4b31-a064-8b5811a9e185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.706157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67065c-a767-4b31-a064-8b5811a9e185 admin admin] PATH_INFO: `/v3/projects/621e741a67af464a9d16641a24c13bdc/users/939a3320873f4a1db47ea0b23da427ea/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.708145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67065c-a767-4b31-a064-8b5811a9e185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=621e741a67af464a9d16641a24c13bdc, user_id=939a3320873f4a1db47ea0b23da427ea, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.710527 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49316e2b-14dd-4599-bcb9-a5cb79773afb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.710812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67065c-a767-4b31-a064-8b5811a9e185 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.717653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb80a5a-e4f1-417d-a540-5c8356211275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.718350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb80a5a-e4f1-417d-a540-5c8356211275 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.718636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb80a5a-e4f1-417d-a540-5c8356211275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.718990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb80a5a-e4f1-417d-a540-5c8356211275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.721678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb80a5a-e4f1-417d-a540-5c8356211275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.723320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb80a5a-e4f1-417d-a540-5c8356211275 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.723457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1025/2060] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:38 2022] PUT /identity/v3/projects/621e741a67af464a9d16641a24c13bdc/users/939a3320873f4a1db47ea0b23da427ea/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:38.734429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc299d4-f8e0-43f6-9ff7-a06738987f6c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.734725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc299d4-f8e0-43f6-9ff7-a06738987f6c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.735014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc299d4-f8e0-43f6-9ff7-a06738987f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.739592 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1036/2061] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:38 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:38.743993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb80a5a-e4f1-417d-a540-5c8356211275 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.756432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb80a5a-e4f1-417d-a540-5c8356211275 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:38.793839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dbc299d4-f8e0-43f6-9ff7-a06738987f6c None None] MFA Rules not processed for user `939a3320873f4a1db47ea0b23da427ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:38.795415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dbc299d4-f8e0-43f6-9ff7-a06738987f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:38.842426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa25583-f7d8-4c2b-a07b-0eae6b8b0f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:38.843608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa25583-f7d8-4c2b-a07b-0eae6b8b0f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.843960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa25583-f7d8-4c2b-a07b-0eae6b8b0f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.844882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa25583-f7d8-4c2b-a07b-0eae6b8b0f4f admin admin] PATH_INFO: `/v3/projects/3b47a5d24dc14c16a5b56a3fdaaf7c04/users/9eda9e2559f54efa9b4c346b922a438b/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.867524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa25583-f7d8-4c2b-a07b-0eae6b8b0f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b47a5d24dc14c16a5b56a3fdaaf7c04, user_id=9eda9e2559f54efa9b4c346b922a438b, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:38.871241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa25583-f7d8-4c2b-a07b-0eae6b8b0f4f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:38.880715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:38.881119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1026/2062] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:38.890635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1037/2063] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:38 2022] PUT /identity/v3/projects/3b47a5d24dc14c16a5b56a3fdaaf7c04/users/9eda9e2559f54efa9b4c346b922a438b/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:38.894865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc299d4-f8e0-43f6-9ff7-a06738987f6c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:38.896301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dbc299d4-f8e0-43f6-9ff7-a06738987f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:38.903862 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd7146-8da9-40e7-96b4-e5e3337cf58d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:38.904188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd7146-8da9-40e7-96b4-e5e3337cf58d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:38.904534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd7146-8da9-40e7-96b4-e5e3337cf58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:38.951792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-adfd7146-8da9-40e7-96b4-e5e3337cf58d None None] MFA Rules not processed for user `9eda9e2559f54efa9b4c346b922a438b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:38.954980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-adfd7146-8da9-40e7-96b4-e5e3337cf58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:38.992100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc299d4-f8e0-43f6-9ff7-a06738987f6c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:39.028913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:39.069693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1038/2064] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:39.075552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd7146-8da9-40e7-96b4-e5e3337cf58d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:39.091577 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf4c37-4159-4413-9afe-12c78f5a5b23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:39.092257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf4c37-4159-4413-9afe-12c78f5a5b23 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:39.092540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf4c37-4159-4413-9afe-12c78f5a5b23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:39.092809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf4c37-4159-4413-9afe-12c78f5a5b23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:39.184835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf4c37-4159-4413-9afe-12c78f5a5b23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:39.186399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf4c37-4159-4413-9afe-12c78f5a5b23 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:39.206941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1027/2065] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:39.218356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd7146-8da9-40e7-96b4-e5e3337cf58d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:39.226509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d311c83-ffef-4d95-8baf-fee617e7b661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:39.227221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d311c83-ffef-4d95-8baf-fee617e7b661 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:39.227628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d311c83-ffef-4d95-8baf-fee617e7b661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:39.227983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d311c83-ffef-4d95-8baf-fee617e7b661 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:39.228710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d311c83-ffef-4d95-8baf-fee617e7b661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:39.232684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d311c83-ffef-4d95-8baf-fee617e7b661 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:39.286416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1039/2066] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:39 2022] POST /identity/v3/projects => generated 386 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:39.291252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf4c37-4159-4413-9afe-12c78f5a5b23 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:39.309316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf4c37-4159-4413-9afe-12c78f5a5b23 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:39.421933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2adfdd-e027-45ef-bb89-dbcf64ca49ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:39.422821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2adfdd-e027-45ef-bb89-dbcf64ca49ed admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:39.423237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2adfdd-e027-45ef-bb89-dbcf64ca49ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:39.423531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2adfdd-e027-45ef-bb89-dbcf64ca49ed admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:39.424174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2adfdd-e027-45ef-bb89-dbcf64ca49ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:39.427699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2adfdd-e027-45ef-bb89-dbcf64ca49ed admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:39.465135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1028/2067] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:39 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:39.469388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d311c83-ffef-4d95-8baf-fee617e7b661 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:39.488662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d311c83-ffef-4d95-8baf-fee617e7b661 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:39.496396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73e23a-1787-48a0-80df-e59da187553d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:39.497325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73e23a-1787-48a0-80df-e59da187553d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:39.498151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73e23a-1787-48a0-80df-e59da187553d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:39.498667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73e23a-1787-48a0-80df-e59da187553d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:39.500408 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73e23a-1787-48a0-80df-e59da187553d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:39.502783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73e23a-1787-48a0-80df-e59da187553d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:39.516923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1040/2068] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:39 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:39.523295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2adfdd-e027-45ef-bb89-dbcf64ca49ed admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:39.546030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2adfdd-e027-45ef-bb89-dbcf64ca49ed admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:39.575774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1daa9-a46d-451f-a504-67f46f8e1e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:39.576508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1daa9-a46d-451f-a504-67f46f8e1e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:39.577353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1daa9-a46d-451f-a504-67f46f8e1e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:39.582098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1daa9-a46d-451f-a504-67f46f8e1e5a admin admin] PATH_INFO: `/v3/projects/7b02e862098a4377b3327e69a7e1eeaf/users/45c4302907d8405bbec32629ce7bb6f0/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:39.625068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1daa9-a46d-451f-a504-67f46f8e1e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b02e862098a4377b3327e69a7e1eeaf, user_id=45c4302907d8405bbec32629ce7bb6f0, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:39.628905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1daa9-a46d-451f-a504-67f46f8e1e5a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:39.646181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1029/2069] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:39 2022] PUT /identity/v3/projects/7b02e862098a4377b3327e69a7e1eeaf/users/45c4302907d8405bbec32629ce7bb6f0/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:39.655052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73e23a-1787-48a0-80df-e59da187553d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:39.686617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73e23a-1787-48a0-80df-e59da187553d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:39.942888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75f3ef-d1f9-4c8e-ae8e-65ee96f085c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:39.945979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75f3ef-d1f9-4c8e-ae8e-65ee96f085c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:39.947913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75f3ef-d1f9-4c8e-ae8e-65ee96f085c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:39.949602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75f3ef-d1f9-4c8e-ae8e-65ee96f085c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:39.952112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75f3ef-d1f9-4c8e-ae8e-65ee96f085c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:39.960341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75f3ef-d1f9-4c8e-ae8e-65ee96f085c6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:39.971777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:40.037698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1041/2070] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:39 2022] GET /identity/v3/roles => generated 1427 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:40.041397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1daa9-a46d-451f-a504-67f46f8e1e5a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:40.052773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1daa9-a46d-451f-a504-67f46f8e1e5a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:40.067601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3212675-0827-4013-8b33-34ea7f5b8e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:40.068452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3212675-0827-4013-8b33-34ea7f5b8e8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:40.068940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3212675-0827-4013-8b33-34ea7f5b8e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:40.069221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3212675-0827-4013-8b33-34ea7f5b8e8f admin admin] PATH_INFO: `/v3/projects/7b02e862098a4377b3327e69a7e1eeaf/users/45c4302907d8405bbec32629ce7bb6f0/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:40.071673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3212675-0827-4013-8b33-34ea7f5b8e8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b02e862098a4377b3327e69a7e1eeaf, user_id=45c4302907d8405bbec32629ce7bb6f0, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:40.090718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3212675-0827-4013-8b33-34ea7f5b8e8f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:40.108129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1030/2071] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:40 2022] PUT /identity/v3/projects/7b02e862098a4377b3327e69a7e1eeaf/users/45c4302907d8405bbec32629ce7bb6f0/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:40.119582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf47e9-7138-4a44-b6cf-7cdbd9b8e3d1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:40.119582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf47e9-7138-4a44-b6cf-7cdbd9b8e3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:40.119978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf47e9-7138-4a44-b6cf-7cdbd9b8e3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:40.182002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-47cf47e9-7138-4a44-b6cf-7cdbd9b8e3d1 None None] MFA Rules not processed for user `45c4302907d8405bbec32629ce7bb6f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:40.183276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-47cf47e9-7138-4a44-b6cf-7cdbd9b8e3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:40.379830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1042/2072] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:40 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:40.396242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3212675-0827-4013-8b33-34ea7f5b8e8f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:40.401848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf47e9-7138-4a44-b6cf-7cdbd9b8e3d1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:40.406745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-47cf47e9-7138-4a44-b6cf-7cdbd9b8e3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:40.429316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:40.469340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3212675-0827-4013-8b33-34ea7f5b8e8f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:40.473686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501e169-1fd1-4f97-9b54-96b9383af7bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:40.474289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501e169-1fd1-4f97-9b54-96b9383af7bf service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:40.475066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501e169-1fd1-4f97-9b54-96b9383af7bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:40.475066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501e169-1fd1-4f97-9b54-96b9383af7bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:40.487216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf47e9-7138-4a44-b6cf-7cdbd9b8e3d1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:40.491241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e219d-c3f9-475f-95c6-8532a56a9af3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:40.492410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e219d-c3f9-475f-95c6-8532a56a9af3 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:40.492410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e219d-c3f9-475f-95c6-8532a56a9af3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:40.492410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e219d-c3f9-475f-95c6-8532a56a9af3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:40.495468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8f6e219d-c3f9-475f-95c6-8532a56a9af3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:40.558836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501e169-1fd1-4f97-9b54-96b9383af7bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:40.560282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501e169-1fd1-4f97-9b54-96b9383af7bf service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:40.568238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e219d-c3f9-475f-95c6-8532a56a9af3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:40.569721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e219d-c3f9-475f-95c6-8532a56a9af3 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:40.571918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1031/2073] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:40.580504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1043/2074] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:40.739705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501e169-1fd1-4f97-9b54-96b9383af7bf service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:40.835648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501e169-1fd1-4f97-9b54-96b9383af7bf service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:40.851394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:40.854583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:40.855111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:40.855543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] PATH_INFO: `/v3/users/cfb2b8eccc84420ab59b9ff9d8582d26` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:40.858472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb2b8eccc84420ab59b9ff9d8582d26)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:40.862658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:40.950028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb2b8eccc84420ab59b9ff9d8582d26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:40.959747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb2b8eccc84420ab59b9ff9d8582d26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:40.960783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb2b8eccc84420ab59b9ff9d8582d26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:40.982729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb2b8eccc84420ab59b9ff9d8582d26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:40.988759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:40.992200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb2b8eccc84420ab59b9ff9d8582d26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:41.000706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1032/2075] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:40 2022] DELETE /identity/v3/users/cfb2b8eccc84420ab59b9ff9d8582d26 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:41.005732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca6f5d-bf40-4c88-8f10-6bed8fac7773 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:41.006207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca6f5d-bf40-4c88-8f10-6bed8fac7773 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:41.006570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca6f5d-bf40-4c88-8f10-6bed8fac7773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:41.046598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-53ca6f5d-bf40-4c88-8f10-6bed8fac7773 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:41.050089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-53ca6f5d-bf40-4c88-8f10-6bed8fac7773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:41.113155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:41.177361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1044/2076] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:41.199267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:41.200791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:41.355902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d24995-94d2-4212-b97b-625770184ade admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:41.361518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e6e78-a1ec-492c-98c9-7f771375009e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:41.362493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6e78-a1ec-492c-98c9-7f771375009e service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:41.362963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6e78-a1ec-492c-98c9-7f771375009e service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:41.363415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6e78-a1ec-492c-98c9-7f771375009e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:41.386743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e6e78-a1ec-492c-98c9-7f771375009e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:41.388252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e6e78-a1ec-492c-98c9-7f771375009e service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:41.411076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1033/2077] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:41.501408 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca6f5d-bf40-4c88-8f10-6bed8fac7773 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:41.503830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-53ca6f5d-bf40-4c88-8f10-6bed8fac7773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:41.618302 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca6f5d-bf40-4c88-8f10-6bed8fac7773 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:41.623791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c475f6-56c4-4c73-a5a0-747979ec0fa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:41.624625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c475f6-56c4-4c73-a5a0-747979ec0fa8 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:41.625485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c475f6-56c4-4c73-a5a0-747979ec0fa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:41.625485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c475f6-56c4-4c73-a5a0-747979ec0fa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:41.627354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d6c475f6-56c4-4c73-a5a0-747979ec0fa8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:41.734678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c475f6-56c4-4c73-a5a0-747979ec0fa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:41.736309 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c475f6-56c4-4c73-a5a0-747979ec0fa8 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:41.755700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1045/2078] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:41.905084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e6e78-a1ec-492c-98c9-7f771375009e service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:41.906682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f83e6e78-a1ec-492c-98c9-7f771375009e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:41.926494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e6e78-a1ec-492c-98c9-7f771375009e service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:41.931580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:41.932222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:41.932506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:41.932779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] PATH_INFO: `/v3/projects/ca305d7effb241579d6420284bc66d4e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:41.933959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca305d7effb241579d6420284bc66d4e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:41.937042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:41.998335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca305d7effb241579d6420284bc66d4e', 'request_id': 'req-389b9336-f235-40ae-b3b7-dc4f24f39209'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.009611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca305d7effb241579d6420284bc66d4e', 'request_id': 'req-389b9336-f235-40ae-b3b7-dc4f24f39209'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.020985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca305d7effb241579d6420284bc66d4e', 'request_id': 'req-389b9336-f235-40ae-b3b7-dc4f24f39209'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.039515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] The token cache is being invalidate because project ca305d7effb241579d6420284bc66d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:42.040018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.040685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.042321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1034/2079] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:41 2022] DELETE /identity/v3/projects/ca305d7effb241579d6420284bc66d4e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:42.046890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c475f6-56c4-4c73-a5a0-747979ec0fa8 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:42.059080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c475f6-56c4-4c73-a5a0-747979ec0fa8 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:42.148865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:42.149779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:42.150110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:42.150583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] PATH_INFO: `/v3/users/939a3320873f4a1db47ea0b23da427ea` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:42.152676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=939a3320873f4a1db47ea0b23da427ea)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:42.154849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:42.198140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:42.222710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939a3320873f4a1db47ea0b23da427ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.239993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939a3320873f4a1db47ea0b23da427ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.248717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939a3320873f4a1db47ea0b23da427ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.262378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939a3320873f4a1db47ea0b23da427ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.263033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57c470ec-3053-4472-ad9d-dae5c86c6499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939a3320873f4a1db47ea0b23da427ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:42.271553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1046/2080] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:42 2022] DELETE /identity/v3/users/939a3320873f4a1db47ea0b23da427ea => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:42.702314 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:42.706684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:42.767327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b9336-f235-40ae-b3b7-dc4f24f39209 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:42.898079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:42.899694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:42.900408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:42.900920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] PATH_INFO: `/v3/projects/621e741a67af464a9d16641a24c13bdc` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:42.902740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=621e741a67af464a9d16641a24c13bdc)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:42.908802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:42.990990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:43.061358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621e741a67af464a9d16641a24c13bdc', 'request_id': 'req-18c1becc-c1ec-4507-8657-a3d41c30a5f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:43.071028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621e741a67af464a9d16641a24c13bdc', 'request_id': 'req-18c1becc-c1ec-4507-8657-a3d41c30a5f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:43.081278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621e741a67af464a9d16641a24c13bdc', 'request_id': 'req-18c1becc-c1ec-4507-8657-a3d41c30a5f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:43.100800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] The token cache is being invalidate because project 621e741a67af464a9d16641a24c13bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:43.101168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:43.103994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c1becc-c1ec-4507-8657-a3d41c30a5f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:43.105945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1035/2081] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:42 2022] DELETE /identity/v3/projects/621e741a67af464a9d16641a24c13bdc => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:43.111634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfcebf-408f-4a75-9ef4-d19129c3a045 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:43.112306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfcebf-408f-4a75-9ef4-d19129c3a045 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:43.112650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfcebf-408f-4a75-9ef4-d19129c3a045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:43.133908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f7bfcebf-408f-4a75-9ef4-d19129c3a045 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:43.135178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f7bfcebf-408f-4a75-9ef4-d19129c3a045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:43.226992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1047/2082] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:43.237053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d54cc9-bc1b-4356-b08d-2f8314debf37 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:43.237334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d54cc9-bc1b-4356-b08d-2f8314debf37 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:43.237668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d54cc9-bc1b-4356-b08d-2f8314debf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:43.268137 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-82d54cc9-bc1b-4356-b08d-2f8314debf37 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:43.269260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-82d54cc9-bc1b-4356-b08d-2f8314debf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:43.281356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1036/2083] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:43.285852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfcebf-408f-4a75-9ef4-d19129c3a045 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:43.304164 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfcebf-408f-4a75-9ef4-d19129c3a045 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:43.309964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf49a2-48f0-4cf2-bdbb-dfb103606911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:43.310727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf49a2-48f0-4cf2-bdbb-dfb103606911 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:43.311014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf49a2-48f0-4cf2-bdbb-dfb103606911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:43.311292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf49a2-48f0-4cf2-bdbb-dfb103606911 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:43.311983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf49a2-48f0-4cf2-bdbb-dfb103606911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:43.312289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf49a2-48f0-4cf2-bdbb-dfb103606911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:43.314125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf49a2-48f0-4cf2-bdbb-dfb103606911 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:43.330931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1048/2084] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:43.336165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d54cc9-bc1b-4356-b08d-2f8314debf37 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:43.350252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d54cc9-bc1b-4356-b08d-2f8314debf37 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:43.360523 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151e6bc-c4ea-4998-893b-cf4ab4cc1757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:43.361216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151e6bc-c4ea-4998-893b-cf4ab4cc1757 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:43.361497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151e6bc-c4ea-4998-893b-cf4ab4cc1757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:43.361799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151e6bc-c4ea-4998-893b-cf4ab4cc1757 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:43.362449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151e6bc-c4ea-4998-893b-cf4ab4cc1757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:43.364461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151e6bc-c4ea-4998-893b-cf4ab4cc1757 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:43.396704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1037/2085] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:43 2022] POST /identity/v3/projects => generated 366 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:43.401216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf49a2-48f0-4cf2-bdbb-dfb103606911 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:43.414238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf49a2-48f0-4cf2-bdbb-dfb103606911 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:43.523526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d814653-76e7-46ef-9f2f-58340cb0bdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:43.524050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d814653-76e7-46ef-9f2f-58340cb0bdaa admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:43.524373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d814653-76e7-46ef-9f2f-58340cb0bdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:43.524830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d814653-76e7-46ef-9f2f-58340cb0bdaa admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:43.525722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d814653-76e7-46ef-9f2f-58340cb0bdaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:43.528606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d814653-76e7-46ef-9f2f-58340cb0bdaa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:43.571067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151e6bc-c4ea-4998-893b-cf4ab4cc1757 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:43.573103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1049/2086] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:43 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:43.583118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151e6bc-c4ea-4998-893b-cf4ab4cc1757 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:43.588976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b04f6-4f63-4469-a4e6-3808b9dafcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:43.589623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b04f6-4f63-4469-a4e6-3808b9dafcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:43.589912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b04f6-4f63-4469-a4e6-3808b9dafcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:43.590187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b04f6-4f63-4469-a4e6-3808b9dafcf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:43.590844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b04f6-4f63-4469-a4e6-3808b9dafcf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:43.592472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b04f6-4f63-4469-a4e6-3808b9dafcf8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:43.604875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1038/2087] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:43 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:43.608980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d814653-76e7-46ef-9f2f-58340cb0bdaa admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:43.624191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d814653-76e7-46ef-9f2f-58340cb0bdaa admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:43.629538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf9555-5e20-414b-9359-e2abff2601eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:43.630323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf9555-5e20-414b-9359-e2abff2601eb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:43.630712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf9555-5e20-414b-9359-e2abff2601eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:43.631109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf9555-5e20-414b-9359-e2abff2601eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:43.631895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf9555-5e20-414b-9359-e2abff2601eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:43.633857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf9555-5e20-414b-9359-e2abff2601eb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:43.655073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1050/2088] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:43 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:43.661002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b04f6-4f63-4469-a4e6-3808b9dafcf8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:43.676147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b04f6-4f63-4469-a4e6-3808b9dafcf8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:43.681765 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69f7fe-c605-4261-bbf7-5495b8c88985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:43.682454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69f7fe-c605-4261-bbf7-5495b8c88985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:43.682847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69f7fe-c605-4261-bbf7-5495b8c88985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:43.683155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69f7fe-c605-4261-bbf7-5495b8c88985 admin admin] PATH_INFO: `/v3/projects/28e8a3b022fc43ae80af48ed39dc599a/users/70bcec5bc9564ed8b56ad264a21ff704/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:43.701418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69f7fe-c605-4261-bbf7-5495b8c88985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e8a3b022fc43ae80af48ed39dc599a, user_id=70bcec5bc9564ed8b56ad264a21ff704, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:43.704811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69f7fe-c605-4261-bbf7-5495b8c88985 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:43.722150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1039/2089] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:43 2022] PUT /identity/v3/projects/28e8a3b022fc43ae80af48ed39dc599a/users/70bcec5bc9564ed8b56ad264a21ff704/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:43.735561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bb6fc-6e11-4e6b-ae12-41ac28ffa6a4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:43.736039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bb6fc-6e11-4e6b-ae12-41ac28ffa6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:43.736600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bb6fc-6e11-4e6b-ae12-41ac28ffa6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:43.773189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:43.776427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9a0bb6fc-6e11-4e6b-ae12-41ac28ffa6a4 None None] MFA Rules not processed for user `70bcec5bc9564ed8b56ad264a21ff704`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:43.779673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9a0bb6fc-6e11-4e6b-ae12-41ac28ffa6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:43.873711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1051/2090] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:43.879669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69f7fe-c605-4261-bbf7-5495b8c88985 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:44.005149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69f7fe-c605-4261-bbf7-5495b8c88985 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:44.013016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7d133-1cac-4365-9745-9579edbce131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:44.014100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7d133-1cac-4365-9745-9579edbce131 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:44.014657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7d133-1cac-4365-9745-9579edbce131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:44.015115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7d133-1cac-4365-9745-9579edbce131 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:44.015872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7d133-1cac-4365-9745-9579edbce131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:44.017866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7d133-1cac-4365-9745-9579edbce131 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:44.073193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1040/2091] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:43 2022] POST /identity/v3/projects => generated 366 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:44.078331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bb6fc-6e11-4e6b-ae12-41ac28ffa6a4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:44.096322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bb6fc-6e11-4e6b-ae12-41ac28ffa6a4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:44.220590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85048bf2-831d-422b-b0ad-ffaf7680967a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:44.221956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85048bf2-831d-422b-b0ad-ffaf7680967a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:44.222309 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85048bf2-831d-422b-b0ad-ffaf7680967a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:44.222794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85048bf2-831d-422b-b0ad-ffaf7680967a admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:44.224236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85048bf2-831d-422b-b0ad-ffaf7680967a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:44.227249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85048bf2-831d-422b-b0ad-ffaf7680967a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:44.267251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7d133-1cac-4365-9745-9579edbce131 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:44.273024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1052/2092] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:44 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:44.279160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7d133-1cac-4365-9745-9579edbce131 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:44.284608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d960f82-ac26-4e88-9b60-afe91cc2f0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:44.285360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d960f82-ac26-4e88-9b60-afe91cc2f0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:44.293781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d960f82-ac26-4e88-9b60-afe91cc2f0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:44.294138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d960f82-ac26-4e88-9b60-afe91cc2f0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:44.294908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d960f82-ac26-4e88-9b60-afe91cc2f0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:44.296608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d960f82-ac26-4e88-9b60-afe91cc2f0e9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:44.328312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1041/2093] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:44 2022] GET /identity/v3/roles => generated 1427 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:44.332660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85048bf2-831d-422b-b0ad-ffaf7680967a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:44.345528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85048bf2-831d-422b-b0ad-ffaf7680967a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:44.350697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3b6ea-fdf6-4358-a2bf-075925e8385a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:44.351617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3b6ea-fdf6-4358-a2bf-075925e8385a admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:44.352085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3b6ea-fdf6-4358-a2bf-075925e8385a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:44.353098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3b6ea-fdf6-4358-a2bf-075925e8385a admin admin] PATH_INFO: `/v3/projects/df2c48c3288d4fb498f9470d091543f8/users/af2efa16aaf14e4a864b25a928705e9e/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:44.373177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3b6ea-fdf6-4358-a2bf-075925e8385a admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2c48c3288d4fb498f9470d091543f8, user_id=af2efa16aaf14e4a864b25a928705e9e, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:44.376563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3b6ea-fdf6-4358-a2bf-075925e8385a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:44.400642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1053/2094] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:44 2022] PUT /identity/v3/projects/df2c48c3288d4fb498f9470d091543f8/users/af2efa16aaf14e4a864b25a928705e9e/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:44.405421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d960f82-ac26-4e88-9b60-afe91cc2f0e9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:44.418692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d960f82-ac26-4e88-9b60-afe91cc2f0e9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:44.453619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:44.540354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3b6ea-fdf6-4358-a2bf-075925e8385a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:44.544308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-20b3b6ea-fdf6-4358-a2bf-075925e8385a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:44.552895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a51cc7-1395-4365-91a9-ded2c3b8f344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:44.553855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a51cc7-1395-4365-91a9-ded2c3b8f344 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:44.554295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a51cc7-1395-4365-91a9-ded2c3b8f344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:44.554718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a51cc7-1395-4365-91a9-ded2c3b8f344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:44.555560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a51cc7-1395-4365-91a9-ded2c3b8f344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:44.557380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a51cc7-1395-4365-91a9-ded2c3b8f344 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:44.577867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1042/2095] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:44 2022] GET /identity/v3/roles => generated 1427 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:44.579054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a51cc7-1395-4365-91a9-ded2c3b8f344 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:44.595953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a51cc7-1395-4365-91a9-ded2c3b8f344 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:44.603065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa8d5d-5e5d-48a9-bdbd-7c5daefc7820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:44.604063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa8d5d-5e5d-48a9-bdbd-7c5daefc7820 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:44.604377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa8d5d-5e5d-48a9-bdbd-7c5daefc7820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:44.604651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa8d5d-5e5d-48a9-bdbd-7c5daefc7820 admin admin] PATH_INFO: `/v3/projects/df2c48c3288d4fb498f9470d091543f8/users/af2efa16aaf14e4a864b25a928705e9e/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:44.607285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa8d5d-5e5d-48a9-bdbd-7c5daefc7820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2c48c3288d4fb498f9470d091543f8, user_id=af2efa16aaf14e4a864b25a928705e9e, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:44.610954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa8d5d-5e5d-48a9-bdbd-7c5daefc7820 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:44.630949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1043/2096] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:44 2022] PUT /identity/v3/projects/df2c48c3288d4fb498f9470d091543f8/users/af2efa16aaf14e4a864b25a928705e9e/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:44.645195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbe802-9eef-4273-9231-f40d9eedfc60 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:44.646457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbe802-9eef-4273-9231-f40d9eedfc60 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:44.646813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbe802-9eef-4273-9231-f40d9eedfc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:44.700886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3b6ea-fdf6-4358-a2bf-075925e8385a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:44.704232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-45bbe802-9eef-4273-9231-f40d9eedfc60 None None] MFA Rules not processed for user `af2efa16aaf14e4a864b25a928705e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:44.706127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-45bbe802-9eef-4273-9231-f40d9eedfc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:44.746128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:44.851499 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:44.852711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:44.853086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:44.854620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] PATH_INFO: `/v3/users/982d0778ec684600a315b0cda8c47a8a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:44.855113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=982d0778ec684600a315b0cda8c47a8a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:44.860080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:44.869870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1044/2097] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:44.886448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbe802-9eef-4273-9231-f40d9eedfc60 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:44.898305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-45bbe802-9eef-4273-9231-f40d9eedfc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:44.955654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982d0778ec684600a315b0cda8c47a8a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:44.979116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982d0778ec684600a315b0cda8c47a8a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:45.011837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982d0778ec684600a315b0cda8c47a8a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:45.032787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbe802-9eef-4273-9231-f40d9eedfc60 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:45.037306 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982d0778ec684600a315b0cda8c47a8a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:45.039590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d32564-a988-461a-9115-30d8507b5e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982d0778ec684600a315b0cda8c47a8a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:45.041605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:45.042700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:45.043063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:45.043402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:45.047145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:45.056490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1054/2098] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:44 2022] DELETE /identity/v3/users/982d0778ec684600a315b0cda8c47a8a => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:45.061695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83434f03-4c75-4053-a5b0-2fc29a0624e4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:45.061985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83434f03-4c75-4053-a5b0-2fc29a0624e4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:45.062356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83434f03-4c75-4053-a5b0-2fc29a0624e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:45.064605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:45.066746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:45.088589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1045/2099] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:45.089812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-83434f03-4c75-4053-a5b0-2fc29a0624e4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:45.091208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-83434f03-4c75-4053-a5b0-2fc29a0624e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:45.213628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1055/2100] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:45.235875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:45.238528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:45.298024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83434f03-4c75-4053-a5b0-2fc29a0624e4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:45.299720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-83434f03-4c75-4053-a5b0-2fc29a0624e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:45.308980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:45.361584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b695e-35d3-43ae-8030-cbcca747a596 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:45.418468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:45.459483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83434f03-4c75-4053-a5b0-2fc29a0624e4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:45.470785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:45.471414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:45.471698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:45.472349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:45.475385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:45.495852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab490a-178f-491e-9a41-425f06bb6151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:45.496498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab490a-178f-491e-9a41-425f06bb6151 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:45.496784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab490a-178f-491e-9a41-425f06bb6151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:45.497062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab490a-178f-491e-9a41-425f06bb6151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:45.521258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab490a-178f-491e-9a41-425f06bb6151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:45.525274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab490a-178f-491e-9a41-425f06bb6151 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:45.560797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1046/2101] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:45.631231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:45.632721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:45.663472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1056/2102] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:45.877760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab490a-178f-491e-9a41-425f06bb6151 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:45.904676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab490a-178f-491e-9a41-425f06bb6151 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:45.919794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee93a0-1fb1-4b6d-aec1-8dd09b4d612b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:45.921019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee93a0-1fb1-4b6d-aec1-8dd09b4d612b service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:45.921476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee93a0-1fb1-4b6d-aec1-8dd09b4d612b service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:45.921861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee93a0-1fb1-4b6d-aec1-8dd09b4d612b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:45.924174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-75ee93a0-1fb1-4b6d-aec1-8dd09b4d612b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:46.101957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee93a0-1fb1-4b6d-aec1-8dd09b4d612b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:46.104344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee93a0-1fb1-4b6d-aec1-8dd09b4d612b service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:46.126911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1047/2103] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:45 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:46.152392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:46.153842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:46.177707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503d6fd-6cef-4b93-be68-6d2c3cbd8c1e service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:46.183479 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:46.184214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:46.184493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:46.184764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] PATH_INFO: `/v3/projects/04c8ea419f39467690de6c5f6056cebe` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:46.185938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c8ea419f39467690de6c5f6056cebe)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:46.188043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:46.248237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c8ea419f39467690de6c5f6056cebe', 'request_id': 'req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.260333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c8ea419f39467690de6c5f6056cebe', 'request_id': 'req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.274216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c8ea419f39467690de6c5f6056cebe', 'request_id': 'req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.298092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] The token cache is being invalidate because project 04c8ea419f39467690de6c5f6056cebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:46.298935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.298935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.303672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1057/2104] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:46 2022] DELETE /identity/v3/projects/04c8ea419f39467690de6c5f6056cebe => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:46.304938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee93a0-1fb1-4b6d-aec1-8dd09b4d612b service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:46.317120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee93a0-1fb1-4b6d-aec1-8dd09b4d612b service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:46.419124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:46.422942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:46.423237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:46.423644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] PATH_INFO: `/v3/users/34e04c5c531f4890b4bffeee35cfc4da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:46.425439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e04c5c531f4890b4bffeee35cfc4da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:46.427670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:46.473375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:46.511059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e04c5c531f4890b4bffeee35cfc4da'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.521175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e04c5c531f4890b4bffeee35cfc4da'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.527621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e04c5c531f4890b4bffeee35cfc4da'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.556437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e04c5c531f4890b4bffeee35cfc4da'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.568981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1abc9b19-c47c-4aca-9ee3-d3f6a9476e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e04c5c531f4890b4bffeee35cfc4da'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.581024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1048/2105] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:46 2022] DELETE /identity/v3/users/34e04c5c531f4890b4bffeee35cfc4da => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:46.595246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:46.598174 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:46.619744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68139e90-4e5b-4f88-8bd9-696ef1a4dad0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:46.712558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:46.713208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:46.713658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:46.713905 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] PATH_INFO: `/v3/users/fc605668580b4de481fcc4e0f13694ae` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:46.715373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc605668580b4de481fcc4e0f13694ae)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:46.717742 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:46.735912 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:46.789512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc605668580b4de481fcc4e0f13694ae'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.809276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc605668580b4de481fcc4e0f13694ae'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.817483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc605668580b4de481fcc4e0f13694ae'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.831112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc605668580b4de481fcc4e0f13694ae'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.831771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc605668580b4de481fcc4e0f13694ae'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:46.850958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1058/2106] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:46 2022] DELETE /identity/v3/users/fc605668580b4de481fcc4e0f13694ae => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:46.856001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbd52a-5ac5-4bc2-9032-55f87eb30c61 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:46.856626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbd52a-5ac5-4bc2-9032-55f87eb30c61 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:46.857007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbd52a-5ac5-4bc2-9032-55f87eb30c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:46.880824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-65fbd52a-5ac5-4bc2-9032-55f87eb30c61 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:46.882230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-65fbd52a-5ac5-4bc2-9032-55f87eb30c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:46.912035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:46.913854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:46.955967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1049/2107] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:46.964921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbd52a-5ac5-4bc2-9032-55f87eb30c61 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:46.966896 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-65fbd52a-5ac5-4bc2-9032-55f87eb30c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:47.040970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa703f4-2716-4984-9406-394cd66aca13 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.045854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbd52a-5ac5-4bc2-9032-55f87eb30c61 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.052321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.054247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.054611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.054951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] PATH_INFO: `/v3/projects/654f9f4c047142deb95bf60f01f3f0fd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.057351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] RBAC: Authorizing `identity:delete_project(project_id=654f9f4c047142deb95bf60f01f3f0fd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:47.059644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:47.064917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3de8a-a49d-4ae5-8661-eead9ba4ad5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.065982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3de8a-a49d-4ae5-8661-eead9ba4ad5c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.066465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3de8a-a49d-4ae5-8661-eead9ba4ad5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.067240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3de8a-a49d-4ae5-8661-eead9ba4ad5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.080647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3de8a-a49d-4ae5-8661-eead9ba4ad5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:47.082910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3de8a-a49d-4ae5-8661-eead9ba4ad5c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:47.096271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1050/2108] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:47.161487 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654f9f4c047142deb95bf60f01f3f0fd', 'request_id': 'req-49cde507-71e9-4894-8780-c7ee7116f36d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.172196 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654f9f4c047142deb95bf60f01f3f0fd', 'request_id': 'req-49cde507-71e9-4894-8780-c7ee7116f36d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.179856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654f9f4c047142deb95bf60f01f3f0fd', 'request_id': 'req-49cde507-71e9-4894-8780-c7ee7116f36d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.193899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] The token cache is being invalidate because project 654f9f4c047142deb95bf60f01f3f0fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:47.194203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.194971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-49cde507-71e9-4894-8780-c7ee7116f36d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.196964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1059/2109] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:46 2022] DELETE /identity/v3/projects/654f9f4c047142deb95bf60f01f3f0fd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:47.203128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc37b0-22f9-45db-9164-f83766eaf6ef None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.204036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc37b0-22f9-45db-9164-f83766eaf6ef None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.204390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc37b0-22f9-45db-9164-f83766eaf6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.228546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-12dc37b0-22f9-45db-9164-f83766eaf6ef None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:47.230140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-12dc37b0-22f9-45db-9164-f83766eaf6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:47.293598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:47.324129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1051/2110] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:47.332143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be2e79-0845-4ed2-aca0-904a75a8f21a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.332696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be2e79-0845-4ed2-aca0-904a75a8f21a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.332696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be2e79-0845-4ed2-aca0-904a75a8f21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.362731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-22be2e79-0845-4ed2-aca0-904a75a8f21a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:47.365537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-22be2e79-0845-4ed2-aca0-904a75a8f21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:47.378993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1060/2111] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:47.383282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc37b0-22f9-45db-9164-f83766eaf6ef None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:47.394147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc37b0-22f9-45db-9164-f83766eaf6ef None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.400062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ec102-3439-48f3-b926-5c533d3da506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.400933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ec102-3439-48f3-b926-5c533d3da506 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.401399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ec102-3439-48f3-b926-5c533d3da506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.401478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ec102-3439-48f3-b926-5c533d3da506 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.402204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ec102-3439-48f3-b926-5c533d3da506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:47.402511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ec102-3439-48f3-b926-5c533d3da506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:47.404156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ec102-3439-48f3-b926-5c533d3da506 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:47.428352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be2e79-0845-4ed2-aca0-904a75a8f21a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:47.429799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-22be2e79-0845-4ed2-aca0-904a75a8f21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:47.444123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1052/2112] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:47.445893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ec102-3439-48f3-b926-5c533d3da506 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:47.458286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ec102-3439-48f3-b926-5c533d3da506 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.461449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be2e79-0845-4ed2-aca0-904a75a8f21a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.464676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f831d5-d781-4305-8cb4-9bead29e4a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.465275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f831d5-d781-4305-8cb4-9bead29e4a49 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.465674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f831d5-d781-4305-8cb4-9bead29e4a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.465957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f831d5-d781-4305-8cb4-9bead29e4a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.466598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f831d5-d781-4305-8cb4-9bead29e4a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:47.468371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.468694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f831d5-d781-4305-8cb4-9bead29e4a49 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:47.469007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.469287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.469558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] PATH_INFO: `/v3/projects/a930aab479e64135bdcdde16ea3b0a68` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.470785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a930aab479e64135bdcdde16ea3b0a68)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:47.473084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:47.491333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:47.519506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1053/2113] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:47 2022] POST /identity/v3/projects => generated 408 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:47.523487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f831d5-d781-4305-8cb4-9bead29e4a49 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:47.545506 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a930aab479e64135bdcdde16ea3b0a68', 'request_id': 'req-ec219831-9abe-4d50-8d33-0e6ab3e4d541'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.555426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a930aab479e64135bdcdde16ea3b0a68', 'request_id': 'req-ec219831-9abe-4d50-8d33-0e6ab3e4d541'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.564546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a930aab479e64135bdcdde16ea3b0a68', 'request_id': 'req-ec219831-9abe-4d50-8d33-0e6ab3e4d541'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.588948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] The token cache is being invalidate because project a930aab479e64135bdcdde16ea3b0a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:47.589617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.590518 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.597650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1061/2114] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:47 2022] DELETE /identity/v3/projects/a930aab479e64135bdcdde16ea3b0a68 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:47.599010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:47.600592 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:47.612719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f831d5-d781-4305-8cb4-9bead29e4a49 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.696738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec219831-9abe-4d50-8d33-0e6ab3e4d541 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.701626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf0b1c-c921-48c2-b2b9-514a8833c52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.702549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0b1c-c921-48c2-b2b9-514a8833c52d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.702549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0b1c-c921-48c2-b2b9-514a8833c52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.702760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.702927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0b1c-c921-48c2-b2b9-514a8833c52d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.703418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf0b1c-c921-48c2-b2b9-514a8833c52d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:47.703556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.704141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.704574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] PATH_INFO: `/v3/users/7b6dea47df884eb4ab7885e0622f670d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.705406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf0b1c-c921-48c2-b2b9-514a8833c52d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:47.706017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6dea47df884eb4ab7885e0622f670d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:47.708280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:47.744202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1054/2115] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:47 2022] POST /identity/v3/users => generated 305 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:47.748587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf0b1c-c921-48c2-b2b9-514a8833c52d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:47.750447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-acaf0b1c-c921-48c2-b2b9-514a8833c52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:47.765678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf0b1c-c921-48c2-b2b9-514a8833c52d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.769911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6dea47df884eb4ab7885e0622f670d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.770456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3fad5-c722-4b8e-affa-d723fe219e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.771177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3fad5-c722-4b8e-affa-d723fe219e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.771455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3fad5-c722-4b8e-affa-d723fe219e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.771842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3fad5-c722-4b8e-affa-d723fe219e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.772515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3fad5-c722-4b8e-affa-d723fe219e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:47.774959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3fad5-c722-4b8e-affa-d723fe219e1c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:47.787410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6dea47df884eb4ab7885e0622f670d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.791287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1055/2116] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:47 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:47.792795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3fad5-c722-4b8e-affa-d723fe219e1c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:47.802724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6dea47df884eb4ab7885e0622f670d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.813353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3fad5-c722-4b8e-affa-d723fe219e1c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.815265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6dea47df884eb4ab7885e0622f670d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.815980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6dea47df884eb4ab7885e0622f670d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:47.819081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719d8bc-d65f-43b8-8bdb-5287d9bd40dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.819608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d8bc-d65f-43b8-8bdb-5287d9bd40dc admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.819897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d8bc-d65f-43b8-8bdb-5287d9bd40dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.820298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d8bc-d65f-43b8-8bdb-5287d9bd40dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:47.821085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719d8bc-d65f-43b8-8bdb-5287d9bd40dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:47.822860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719d8bc-d65f-43b8-8bdb-5287d9bd40dc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:47.826638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1062/2117] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:47 2022] DELETE /identity/v3/users/7b6dea47df884eb4ab7885e0622f670d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:47.834391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1056/2118] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:47 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:47.838693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:47.841319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:47.890130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:47.974823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bb23d-b5bb-4a9b-a9da-db88b50d7815 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:47.981292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875e732-a8cb-4f82-a6b3-5e3ee7d9a12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:47.982021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875e732-a8cb-4f82-a6b3-5e3ee7d9a12c admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:47.993615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875e732-a8cb-4f82-a6b3-5e3ee7d9a12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:47.993615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875e732-a8cb-4f82-a6b3-5e3ee7d9a12c admin admin] PATH_INFO: `/v3/projects/1f3c298dfe9b49a084f1cfd602ffd609/users/daff4a5dca684cc080f5cf0140202eff/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.003727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875e732-a8cb-4f82-a6b3-5e3ee7d9a12c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f3c298dfe9b49a084f1cfd602ffd609, user_id=daff4a5dca684cc080f5cf0140202eff, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.007224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875e732-a8cb-4f82-a6b3-5e3ee7d9a12c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.028259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1063/2119] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:47 2022] PUT /identity/v3/projects/1f3c298dfe9b49a084f1cfd602ffd609/users/daff4a5dca684cc080f5cf0140202eff/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:48.040486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef5dbe-b3a4-4b5a-943d-c103639c3a2a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.041061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef5dbe-b3a4-4b5a-943d-c103639c3a2a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.041153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef5dbe-b3a4-4b5a-943d-c103639c3a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.076634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-16ef5dbe-b3a4-4b5a-943d-c103639c3a2a None None] MFA Rules not processed for user `daff4a5dca684cc080f5cf0140202eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:48.077301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-16ef5dbe-b3a4-4b5a-943d-c103639c3a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:48.102072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:48.188566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1057/2120] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:48 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:48.193899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875e732-a8cb-4f82-a6b3-5e3ee7d9a12c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:48.265921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875e732-a8cb-4f82-a6b3-5e3ee7d9a12c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:48.270108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef5dbe-b3a4-4b5a-943d-c103639c3a2a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:48.270755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807ab49-9c70-4079-ab62-6da75c9813b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:48.271436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807ab49-9c70-4079-ab62-6da75c9813b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.271745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807ab49-9c70-4079-ab62-6da75c9813b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.272019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807ab49-9c70-4079-ab62-6da75c9813b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.272787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807ab49-9c70-4079-ab62-6da75c9813b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.275861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807ab49-9c70-4079-ab62-6da75c9813b5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.277746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-16ef5dbe-b3a4-4b5a-943d-c103639c3a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:48.293525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef5dbe-b3a4-4b5a-943d-c103639c3a2a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:48.298936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:48.299560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.299866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.300115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] PATH_INFO: `/v3/projects/3c4cf944302f45b18e108e8767a28368` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.301276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4cf944302f45b18e108e8767a28368)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.303251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.319965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1064/2121] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:48 2022] POST /identity/v3/projects => generated 410 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:48.325851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807ab49-9c70-4079-ab62-6da75c9813b5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:48.357484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4cf944302f45b18e108e8767a28368', 'request_id': 'req-aaa34348-1414-409d-9216-b751b667cf34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.364094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4cf944302f45b18e108e8767a28368', 'request_id': 'req-aaa34348-1414-409d-9216-b751b667cf34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.370733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4cf944302f45b18e108e8767a28368', 'request_id': 'req-aaa34348-1414-409d-9216-b751b667cf34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.385423 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] The token cache is being invalidate because project 3c4cf944302f45b18e108e8767a28368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:48.385727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.386440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.388498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1058/2122] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:48 2022] DELETE /identity/v3/projects/3c4cf944302f45b18e108e8767a28368 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:48.392491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:48.393836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:48.400469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807ab49-9c70-4079-ab62-6da75c9813b5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:48.440957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:48.470987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786de95-0fd6-4dc2-b516-c0d8d9924165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:48.471596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786de95-0fd6-4dc2-b516-c0d8d9924165 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.471879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786de95-0fd6-4dc2-b516-c0d8d9924165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.472152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786de95-0fd6-4dc2-b516-c0d8d9924165 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.472785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786de95-0fd6-4dc2-b516-c0d8d9924165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.474753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786de95-0fd6-4dc2-b516-c0d8d9924165 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.474989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34348-1414-409d-9216-b751b667cf34 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:48.481008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:48.481672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.481951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.482220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] PATH_INFO: `/v3/users/dbc8c7d5f213404fab22df19021b3558` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.483381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc8c7d5f213404fab22df19021b3558)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.485274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.498820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1065/2123] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:48 2022] POST /identity/v3/users => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:48.504153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786de95-0fd6-4dc2-b516-c0d8d9924165 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:48.506181 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0786de95-0fd6-4dc2-b516-c0d8d9924165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:48.522354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786de95-0fd6-4dc2-b516-c0d8d9924165 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:48.528543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2351d-e7de-44cf-8739-70c6d6112162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:48.529207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2351d-e7de-44cf-8739-70c6d6112162 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.529484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2351d-e7de-44cf-8739-70c6d6112162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.529778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2351d-e7de-44cf-8739-70c6d6112162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.530443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2351d-e7de-44cf-8739-70c6d6112162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.532216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2351d-e7de-44cf-8739-70c6d6112162 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.542615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:48.545176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc8c7d5f213404fab22df19021b3558'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.545417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1066/2124] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:48 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:48.549620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2351d-e7de-44cf-8739-70c6d6112162 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:48.556615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc8c7d5f213404fab22df19021b3558'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.558808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc8c7d5f213404fab22df19021b3558'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.563560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2351d-e7de-44cf-8739-70c6d6112162 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:48.568711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007434b2-4d1b-4d4d-953f-202ccb63c14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:48.569418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007434b2-4d1b-4d4d-953f-202ccb63c14a admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.569697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007434b2-4d1b-4d4d-953f-202ccb63c14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.569969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007434b2-4d1b-4d4d-953f-202ccb63c14a admin admin] PATH_INFO: `/v3/projects/14a834aa47984964bbb82a8d017b60c5/users/6775b87e75b5484a9b6290127da0cc0b/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.575535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc8c7d5f213404fab22df19021b3558'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.586315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc8c7d5f213404fab22df19021b3558'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:48.590277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007434b2-4d1b-4d4d-953f-202ccb63c14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a834aa47984964bbb82a8d017b60c5, user_id=6775b87e75b5484a9b6290127da0cc0b, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.595294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007434b2-4d1b-4d4d-953f-202ccb63c14a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.596629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1059/2125] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:48 2022] DELETE /identity/v3/users/dbc8c7d5f213404fab22df19021b3558 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:48.624138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1067/2126] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:48 2022] PUT /identity/v3/projects/14a834aa47984964bbb82a8d017b60c5/users/6775b87e75b5484a9b6290127da0cc0b/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:48.630452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:48.633021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:48.755357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923928c-1214-43d0-948a-026f1390eba7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:48.804919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007434b2-4d1b-4d4d-953f-202ccb63c14a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:48.809697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-007434b2-4d1b-4d4d-953f-202ccb63c14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:48.854256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508272e0-f196-467d-b7b1-20ea3ccfb20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:48.855001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508272e0-f196-467d-b7b1-20ea3ccfb20e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.855348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508272e0-f196-467d-b7b1-20ea3ccfb20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.855721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508272e0-f196-467d-b7b1-20ea3ccfb20e admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.856467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508272e0-f196-467d-b7b1-20ea3ccfb20e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.858305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508272e0-f196-467d-b7b1-20ea3ccfb20e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.878652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1060/2127] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:48 2022] GET /identity/v3/roles => generated 1427 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:48.880639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508272e0-f196-467d-b7b1-20ea3ccfb20e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:48.893542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508272e0-f196-467d-b7b1-20ea3ccfb20e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:48.903398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee053f-8465-4e1b-b8fd-6370e05b4b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:48.903740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007434b2-4d1b-4d4d-953f-202ccb63c14a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:48.904194 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee053f-8465-4e1b-b8fd-6370e05b4b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.905097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee053f-8465-4e1b-b8fd-6370e05b4b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.905097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee053f-8465-4e1b-b8fd-6370e05b4b26 admin admin] PATH_INFO: `/v3/projects/14a834aa47984964bbb82a8d017b60c5/users/6775b87e75b5484a9b6290127da0cc0b/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.907150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee053f-8465-4e1b-b8fd-6370e05b4b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a834aa47984964bbb82a8d017b60c5, user_id=6775b87e75b5484a9b6290127da0cc0b, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.909883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee053f-8465-4e1b-b8fd-6370e05b4b26 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.910469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:48.911117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.911403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.912015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] PATH_INFO: `/v3/users/70bcec5bc9564ed8b56ad264a21ff704` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.913371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70bcec5bc9564ed8b56ad264a21ff704)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:48.916543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:48.926519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1061/2128] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:48 2022] PUT /identity/v3/projects/14a834aa47984964bbb82a8d017b60c5/users/6775b87e75b5484a9b6290127da0cc0b/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:48.938863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd25f6-5d89-4b14-8312-06f2509a672f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:48.939159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd25f6-5d89-4b14-8312-06f2509a672f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:48.939333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd25f6-5d89-4b14-8312-06f2509a672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:48.981661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-88bd25f6-5d89-4b14-8312-06f2509a672f None None] MFA Rules not processed for user `6775b87e75b5484a9b6290127da0cc0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:48.982893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-88bd25f6-5d89-4b14-8312-06f2509a672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:48.985458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70bcec5bc9564ed8b56ad264a21ff704'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.003051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70bcec5bc9564ed8b56ad264a21ff704'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.011333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70bcec5bc9564ed8b56ad264a21ff704'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.021172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70bcec5bc9564ed8b56ad264a21ff704'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.021927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70bcec5bc9564ed8b56ad264a21ff704'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.031524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1068/2129] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:48 2022] DELETE /identity/v3/users/70bcec5bc9564ed8b56ad264a21ff704 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:49.032496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.034173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.049542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:49.053783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:49.082421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1062/2130] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:48 2022] POST /identity/v3/auth/tokens => generated 3363 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:49.083717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa441558-3726-49b5-b7a5-e9e1a8553975 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.084029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa441558-3726-49b5-b7a5-e9e1a8553975 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.084245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa441558-3726-49b5-b7a5-e9e1a8553975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.110193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-aa441558-3726-49b5-b7a5-e9e1a8553975 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:49.111513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-aa441558-3726-49b5-b7a5-e9e1a8553975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.125148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea7140-566b-4be3-aa50-1b8f573dcdd5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.132845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:49.133761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.134380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.134493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] PATH_INFO: `/v3/projects/d0db4f8425f34f9792a2638042639695` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.135601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0db4f8425f34f9792a2638042639695)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:49.138789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:49.210096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1063/2131] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:49.211252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa441558-3726-49b5-b7a5-e9e1a8553975 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.214852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-aa441558-3726-49b5-b7a5-e9e1a8553975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.219290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0db4f8425f34f9792a2638042639695', 'request_id': 'req-9ffcb732-c10d-4285-92f9-cdc354827bea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.233421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0db4f8425f34f9792a2638042639695', 'request_id': 'req-9ffcb732-c10d-4285-92f9-cdc354827bea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.242099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0db4f8425f34f9792a2638042639695', 'request_id': 'req-9ffcb732-c10d-4285-92f9-cdc354827bea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.256512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] The token cache is being invalidate because project d0db4f8425f34f9792a2638042639695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:49.256905 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.257793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.261756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1069/2132] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:49 2022] DELETE /identity/v3/projects/d0db4f8425f34f9792a2638042639695 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:49.263051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.265406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.299679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa441558-3726-49b5-b7a5-e9e1a8553975 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.343919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcb732-c10d-4285-92f9-cdc354827bea admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.351536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78facd-b449-4504-acf0-dadf946f023c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:49.352358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78facd-b449-4504-acf0-dadf946f023c service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.352848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78facd-b449-4504-acf0-dadf946f023c service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.353132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78facd-b449-4504-acf0-dadf946f023c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.355799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2f78facd-b449-4504-acf0-dadf946f023c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.358944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4e214-2940-47ce-96b1-e289c727911a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:49.359649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4e214-2940-47ce-96b1-e289c727911a service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.359928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4e214-2940-47ce-96b1-e289c727911a service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.360279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4e214-2940-47ce-96b1-e289c727911a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.362189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-63f4e214-2940-47ce-96b1-e289c727911a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.416311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:49.421164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:49.434510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78facd-b449-4504-acf0-dadf946f023c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:49.436460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78facd-b449-4504-acf0-dadf946f023c service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:49.439574 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4e214-2940-47ce-96b1-e289c727911a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:49.441005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4e214-2940-47ce-96b1-e289c727911a service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:49.448494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1070/2133] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:49.449673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c5a60-34e6-46f6-ad49-0a1489975359 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.449964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c5a60-34e6-46f6-ad49-0a1489975359 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.450320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c5a60-34e6-46f6-ad49-0a1489975359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.452160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1064/2134] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:49 2022] GET /identity/v3/auth/tokens => generated 3363 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:49.473804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-730c5a60-34e6-46f6-ad49-0a1489975359 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:49.475086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-730c5a60-34e6-46f6-ad49-0a1489975359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.485087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1071/2135] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:49.493807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24c60f-de9f-4330-9f82-fbd326d881b6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.494617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24c60f-de9f-4330-9f82-fbd326d881b6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.494617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24c60f-de9f-4330-9f82-fbd326d881b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.532408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ec24c60f-de9f-4330-9f82-fbd326d881b6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:49.534322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ec24c60f-de9f-4330-9f82-fbd326d881b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.544444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1065/2136] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:49.548853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c5a60-34e6-46f6-ad49-0a1489975359 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.560568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c5a60-34e6-46f6-ad49-0a1489975359 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.566222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0217c-0716-4bce-87c0-9889577b8d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:49.566834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0217c-0716-4bce-87c0-9889577b8d6e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.567118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0217c-0716-4bce-87c0-9889577b8d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.567662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0217c-0716-4bce-87c0-9889577b8d6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.568029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0217c-0716-4bce-87c0-9889577b8d6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:49.568336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0217c-0716-4bce-87c0-9889577b8d6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:49.570286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0217c-0716-4bce-87c0-9889577b8d6e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:49.586781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1072/2137] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:49.591809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24c60f-de9f-4330-9f82-fbd326d881b6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.601871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24c60f-de9f-4330-9f82-fbd326d881b6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.607254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157ff9e-c36e-4782-9220-2033d4d5b7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:49.607863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ff9e-c36e-4782-9220-2033d4d5b7fe admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.608611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ff9e-c36e-4782-9220-2033d4d5b7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.608611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ff9e-c36e-4782-9220-2033d4d5b7fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.609034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157ff9e-c36e-4782-9220-2033d4d5b7fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:49.611000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157ff9e-c36e-4782-9220-2033d4d5b7fe admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:49.652194 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1066/2138] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:49 2022] POST /identity/v3/projects => generated 374 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:49.659065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0217c-0716-4bce-87c0-9889577b8d6e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.668960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157ff9e-c36e-4782-9220-2033d4d5b7fe admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.674725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1157ff9e-c36e-4782-9220-2033d4d5b7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.678036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0217c-0716-4bce-87c0-9889577b8d6e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.761047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157ff9e-c36e-4782-9220-2033d4d5b7fe admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.765278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9150d-ac04-4c08-8cdb-408a69d990e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:49.766123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9150d-ac04-4c08-8cdb-408a69d990e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.766406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9150d-ac04-4c08-8cdb-408a69d990e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.766510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c43077-e073-45fe-9dc6-211e5dea51ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:49.766690 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9150d-ac04-4c08-8cdb-408a69d990e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.767485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c43077-e073-45fe-9dc6-211e5dea51ae admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.767984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c43077-e073-45fe-9dc6-211e5dea51ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.768473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c43077-e073-45fe-9dc6-211e5dea51ae admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.768564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b4f9150d-ac04-4c08-8cdb-408a69d990e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.769713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c43077-e073-45fe-9dc6-211e5dea51ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:49.773520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c43077-e073-45fe-9dc6-211e5dea51ae admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:49.818310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1073/2139] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:49 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:49.818984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c43077-e073-45fe-9dc6-211e5dea51ae admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.822182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-39c43077-e073-45fe-9dc6-211e5dea51ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.838709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c43077-e073-45fe-9dc6-211e5dea51ae admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.846716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:49.847779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.848284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.848768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] PATH_INFO: `/v3/projects/28e8a3b022fc43ae80af48ed39dc599a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.850666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e8a3b022fc43ae80af48ed39dc599a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:49.854801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:49.857951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9150d-ac04-4c08-8cdb-408a69d990e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:49.859862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9150d-ac04-4c08-8cdb-408a69d990e4 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:49.877838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1067/2140] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:49 2022] GET /identity/v3/auth/tokens => generated 3294 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:49.878636 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9150d-ac04-4c08-8cdb-408a69d990e4 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.894936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9150d-ac04-4c08-8cdb-408a69d990e4 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.902109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193d275-9380-452d-a4c8-c314c16d3d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:49.902732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193d275-9380-452d-a4c8-c314c16d3d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:49.903022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193d275-9380-452d-a4c8-c314c16d3d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:49.903283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193d275-9380-452d-a4c8-c314c16d3d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:49.903930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193d275-9380-452d-a4c8-c314c16d3d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:49.905662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193d275-9380-452d-a4c8-c314c16d3d7f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:49.920868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1068/2141] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:49 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:49.922592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193d275-9380-452d-a4c8-c314c16d3d7f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.925652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e8a3b022fc43ae80af48ed39dc599a', 'request_id': 'req-0e791b37-1c83-45c7-9937-7d3dae78007b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.934700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193d275-9380-452d-a4c8-c314c16d3d7f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:49.935517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e8a3b022fc43ae80af48ed39dc599a', 'request_id': 'req-0e791b37-1c83-45c7-9937-7d3dae78007b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.942880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e8a3b022fc43ae80af48ed39dc599a', 'request_id': 'req-0e791b37-1c83-45c7-9937-7d3dae78007b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.957987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] The token cache is being invalidate because project 28e8a3b022fc43ae80af48ed39dc599a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:49.958279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.959005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:49.960536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1074/2142] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:49 2022] DELETE /identity/v3/projects/28e8a3b022fc43ae80af48ed39dc599a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:49.964498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:49.966060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:49.997591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:50.006129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa33a8-709a-4602-92ab-e9cbb550f329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:50.006327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa33a8-709a-4602-92ab-e9cbb550f329 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:50.006518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa33a8-709a-4602-92ab-e9cbb550f329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:50.006800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa33a8-709a-4602-92ab-e9cbb550f329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:50.007794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa33a8-709a-4602-92ab-e9cbb550f329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:50.009879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa33a8-709a-4602-92ab-e9cbb550f329 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:50.020182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:50.023681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1069/2143] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:49 2022] GET /identity/v3/roles => generated 1427 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:50.028187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa33a8-709a-4602-92ab-e9cbb550f329 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:50.030508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ecaa33a8-709a-4602-92ab-e9cbb550f329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:50.046992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa33a8-709a-4602-92ab-e9cbb550f329 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:50.053262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d276c-749c-4cc9-ae31-4a30f549c41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:50.054348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d276c-749c-4cc9-ae31-4a30f549c41f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:50.054527 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d276c-749c-4cc9-ae31-4a30f549c41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:50.054957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d276c-749c-4cc9-ae31-4a30f549c41f admin admin] PATH_INFO: `/v3/projects/356d731e34b24c9d8852516ae9141124/users/75de510214004bbeab187d65bf178735/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:50.075546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e791b37-1c83-45c7-9937-7d3dae78007b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:50.087161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d276c-749c-4cc9-ae31-4a30f549c41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=356d731e34b24c9d8852516ae9141124, user_id=75de510214004bbeab187d65bf178735, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:50.089709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:50.091233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:50.091884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:50.092886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d276c-749c-4cc9-ae31-4a30f549c41f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:50.093327 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] PATH_INFO: `/v3/users/af2efa16aaf14e4a864b25a928705e9e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:50.095839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2efa16aaf14e4a864b25a928705e9e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:50.100248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:50.119944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1070/2144] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:50 2022] PUT /identity/v3/projects/356d731e34b24c9d8852516ae9141124/users/75de510214004bbeab187d65bf178735/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:50.126465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ecb80-aa52-4d95-9f46-fc7b0f2ca81e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:50.127075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ecb80-aa52-4d95-9f46-fc7b0f2ca81e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:50.127646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ecb80-aa52-4d95-9f46-fc7b0f2ca81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:50.168197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-930ecb80-aa52-4d95-9f46-fc7b0f2ca81e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:50.169602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-930ecb80-aa52-4d95-9f46-fc7b0f2ca81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:50.224988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2efa16aaf14e4a864b25a928705e9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.249161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1071/2145] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:50.252042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2efa16aaf14e4a864b25a928705e9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.256313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ecb80-aa52-4d95-9f46-fc7b0f2ca81e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:50.259094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-930ecb80-aa52-4d95-9f46-fc7b0f2ca81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:50.264891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2efa16aaf14e4a864b25a928705e9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.275930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2efa16aaf14e4a864b25a928705e9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.276790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2efa16aaf14e4a864b25a928705e9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.287902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1075/2146] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:49 2022] DELETE /identity/v3/users/af2efa16aaf14e4a864b25a928705e9e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:50.371109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ecb80-aa52-4d95-9f46-fc7b0f2ca81e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:50.453470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f5f4b-9763-458f-a1f9-7517be21087d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:50.454102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f5f4b-9763-458f-a1f9-7517be21087d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:50.454382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f5f4b-9763-458f-a1f9-7517be21087d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:50.454648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f5f4b-9763-458f-a1f9-7517be21087d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:50.456390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-eb8f5f4b-9763-458f-a1f9-7517be21087d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:50.502178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:50.544667 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f5f4b-9763-458f-a1f9-7517be21087d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:50.548133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f5f4b-9763-458f-a1f9-7517be21087d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:50.566942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1072/2147] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:50.689521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:50.691535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:50.713710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1915ba3-0372-496f-acf8-91cc8517ef88 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:50.719979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:50.720815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:50.721100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:50.721379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] PATH_INFO: `/v3/projects/df2c48c3288d4fb498f9470d091543f8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:50.722691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2c48c3288d4fb498f9470d091543f8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:50.724525 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:50.787068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2c48c3288d4fb498f9470d091543f8', 'request_id': 'req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.799087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2c48c3288d4fb498f9470d091543f8', 'request_id': 'req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.807088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2c48c3288d4fb498f9470d091543f8', 'request_id': 'req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.819163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] The token cache is being invalidate because project df2c48c3288d4fb498f9470d091543f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:50.819443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.820202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b75076e2-ea6e-4fee-a977-ef8e1c8b8dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:50.822194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1076/2148] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:50 2022] DELETE /identity/v3/projects/df2c48c3288d4fb498f9470d091543f8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:50.826930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31f768-3213-4d73-9819-55d4c8e2f9fa None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:50.827373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31f768-3213-4d73-9819-55d4c8e2f9fa None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:50.827751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31f768-3213-4d73-9819-55d4c8e2f9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:50.851077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-da31f768-3213-4d73-9819-55d4c8e2f9fa None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:50.853546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-da31f768-3213-4d73-9819-55d4c8e2f9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:50.924102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1073/2149] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:50.932699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188308e0-8fe6-49c4-adea-b8b1d07fdf0c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:50.932993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188308e0-8fe6-49c4-adea-b8b1d07fdf0c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:50.933382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188308e0-8fe6-49c4-adea-b8b1d07fdf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:50.954671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-188308e0-8fe6-49c4-adea-b8b1d07fdf0c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:50.955902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-188308e0-8fe6-49c4-adea-b8b1d07fdf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:50.964351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1077/2150] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:50.968642 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31f768-3213-4d73-9819-55d4c8e2f9fa None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:50.979503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31f768-3213-4d73-9819-55d4c8e2f9fa None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:50.984807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcb171-39e3-4bff-809a-19d91dd4e601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:50.985521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcb171-39e3-4bff-809a-19d91dd4e601 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:50.985899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcb171-39e3-4bff-809a-19d91dd4e601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:50.986240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcb171-39e3-4bff-809a-19d91dd4e601 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:50.986968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcb171-39e3-4bff-809a-19d91dd4e601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:50.987342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcb171-39e3-4bff-809a-19d91dd4e601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:50.989145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcb171-39e3-4bff-809a-19d91dd4e601 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.002355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1074/2151] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:51.006920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188308e0-8fe6-49c4-adea-b8b1d07fdf0c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.017430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188308e0-8fe6-49c4-adea-b8b1d07fdf0c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.023899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffea3e-d26a-4c55-887b-dec25af69d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:51.025120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffea3e-d26a-4c55-887b-dec25af69d70 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.025936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffea3e-d26a-4c55-887b-dec25af69d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.026430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffea3e-d26a-4c55-887b-dec25af69d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.027331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffea3e-d26a-4c55-887b-dec25af69d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:51.029564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffea3e-d26a-4c55-887b-dec25af69d70 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.063640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1078/2152] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:51 2022] POST /identity/v3/projects => generated 368 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:37:51.065150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcb171-39e3-4bff-809a-19d91dd4e601 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.078874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcb171-39e3-4bff-809a-19d91dd4e601 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.144885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9196f8-d88d-4fe0-ba7c-f92f6b770ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:51.145488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9196f8-d88d-4fe0-ba7c-f92f6b770ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.145796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9196f8-d88d-4fe0-ba7c-f92f6b770ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.146067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9196f8-d88d-4fe0-ba7c-f92f6b770ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.146692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9196f8-d88d-4fe0-ba7c-f92f6b770ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:51.148612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9196f8-d88d-4fe0-ba7c-f92f6b770ef6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.176053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1075/2153] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:51 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:51.179968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffea3e-d26a-4c55-887b-dec25af69d70 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.191467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffea3e-d26a-4c55-887b-dec25af69d70 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.197046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf6149-77a3-44ae-af25-e8ae34afb4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:51.197764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf6149-77a3-44ae-af25-e8ae34afb4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.198139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf6149-77a3-44ae-af25-e8ae34afb4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.198484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf6149-77a3-44ae-af25-e8ae34afb4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.199279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf6149-77a3-44ae-af25-e8ae34afb4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:51.200953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf6149-77a3-44ae-af25-e8ae34afb4f4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.213416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1079/2154] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:51 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:51.223880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9196f8-d88d-4fe0-ba7c-f92f6b770ef6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.236030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9196f8-d88d-4fe0-ba7c-f92f6b770ef6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.241600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dc3d5-eb98-420a-8778-30a6fe37b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:51.242183 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dc3d5-eb98-420a-8778-30a6fe37b0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.242467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dc3d5-eb98-420a-8778-30a6fe37b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.242798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dc3d5-eb98-420a-8778-30a6fe37b0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.243929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2dc3d5-eb98-420a-8778-30a6fe37b0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:51.246056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2dc3d5-eb98-420a-8778-30a6fe37b0f9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.263050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1076/2155] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:51 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:51.265374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf6149-77a3-44ae-af25-e8ae34afb4f4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.280080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:51.295255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf6149-77a3-44ae-af25-e8ae34afb4f4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.325061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59f14a-5fc7-4f0c-8245-03f31f411829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:51.325957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59f14a-5fc7-4f0c-8245-03f31f411829 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.326318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59f14a-5fc7-4f0c-8245-03f31f411829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.328755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59f14a-5fc7-4f0c-8245-03f31f411829 admin admin] PATH_INFO: `/v3/projects/cec39bcf21204fa2a2bf2f005e23b9df/users/09ceb5de5c9a4540aa3d79bf72ee15f0/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.348431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860303df-dbec-40ad-9744-d7ce03815bbc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.348999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860303df-dbec-40ad-9744-d7ce03815bbc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.349551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860303df-dbec-40ad-9744-d7ce03815bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.363035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59f14a-5fc7-4f0c-8245-03f31f411829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec39bcf21204fa2a2bf2f005e23b9df, user_id=09ceb5de5c9a4540aa3d79bf72ee15f0, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:51.366018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59f14a-5fc7-4f0c-8245-03f31f411829 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.388918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1080/2156] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:51 2022] PUT /identity/v3/projects/cec39bcf21204fa2a2bf2f005e23b9df/users/09ceb5de5c9a4540aa3d79bf72ee15f0/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:51.402522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe117a-1e8f-423c-8242-1ab39971aae1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.403308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe117a-1e8f-423c-8242-1ab39971aae1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.404814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe117a-1e8f-423c-8242-1ab39971aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.409738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-860303df-dbec-40ad-9744-d7ce03815bbc None None] MFA Rules not processed for user `75de510214004bbeab187d65bf178735`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:51.412376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-860303df-dbec-40ad-9744-d7ce03815bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:51.483406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c3fe117a-1e8f-423c-8242-1ab39971aae1 None None] MFA Rules not processed for user `09ceb5de5c9a4540aa3d79bf72ee15f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:51.485882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c3fe117a-1e8f-423c-8242-1ab39971aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:51.521617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:51.591764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1081/2157] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:51.592646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe117a-1e8f-423c-8242-1ab39971aae1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.594234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c3fe117a-1e8f-423c-8242-1ab39971aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:51.601001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1077/2158] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:51.603206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860303df-dbec-40ad-9744-d7ce03815bbc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.723392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe117a-1e8f-423c-8242-1ab39971aae1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.728896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:51.729552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.729906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.730187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] PATH_INFO: `/v3/users/9eda9e2559f54efa9b4c346b922a438b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.731374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eda9e2559f54efa9b4c346b922a438b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:51.733324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.737823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860303df-dbec-40ad-9744-d7ce03815bbc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.743916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5a47a-bb19-4926-a942-9124142b3ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:51.744635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a47a-bb19-4926-a942-9124142b3ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.744991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a47a-bb19-4926-a942-9124142b3ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.746169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a47a-bb19-4926-a942-9124142b3ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.746355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5a47a-bb19-4926-a942-9124142b3ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:51.748612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5a47a-bb19-4926-a942-9124142b3ea7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.779029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1078/2159] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:51 2022] POST /identity/v3/projects => generated 368 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:51.779970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5a47a-bb19-4926-a942-9124142b3ea7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.781651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ddb5a47a-bb19-4926-a942-9124142b3ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:51.796728 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eda9e2559f54efa9b4c346b922a438b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:51.813964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eda9e2559f54efa9b4c346b922a438b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:51.819158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:51.822962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eda9e2559f54efa9b4c346b922a438b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:51.834808 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eda9e2559f54efa9b4c346b922a438b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:51.835577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eda9e2559f54efa9b4c346b922a438b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:51.847805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1082/2160] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:51 2022] DELETE /identity/v3/users/9eda9e2559f54efa9b4c346b922a438b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:51.847805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.847979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5a47a-bb19-4926-a942-9124142b3ea7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.848072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:51.906676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:51.907271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.907551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.907821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.909669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:51.922262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e1c10-bdad-4d07-90c7-a3dee9829f67 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.926947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b9631-eca0-41e1-9903-ec2446be96a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:51.927557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b9631-eca0-41e1-9903-ec2446be96a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.927840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b9631-eca0-41e1-9903-ec2446be96a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.928112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b9631-eca0-41e1-9903-ec2446be96a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.928731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b9631-eca0-41e1-9903-ec2446be96a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:51.930717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b9631-eca0-41e1-9903-ec2446be96a6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.937130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:51.952958 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1083/2161] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:51 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:51.954567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8493205-b398-40b6-8a33-56d4377dd70a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:51.955225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8493205-b398-40b6-8a33-56d4377dd70a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:51.955225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8493205-b398-40b6-8a33-56d4377dd70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:51.974139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:51.974520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d8493205-b398-40b6-8a33-56d4377dd70a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:51.975613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:51.975761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d8493205-b398-40b6-8a33-56d4377dd70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:51.984783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1084/2162] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:51.985295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1079/2163] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:51.985398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8493205-b398-40b6-8a33-56d4377dd70a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.994394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:51.995035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8493205-b398-40b6-8a33-56d4377dd70a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:51.996612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:52.000906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057f694-3be0-4d17-9fac-bbfd2821c9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:52.001824 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057f694-3be0-4d17-9fac-bbfd2821c9ce admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.002264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057f694-3be0-4d17-9fac-bbfd2821c9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.002493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057f694-3be0-4d17-9fac-bbfd2821c9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.003142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057f694-3be0-4d17-9fac-bbfd2821c9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:52.004846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057f694-3be0-4d17-9fac-bbfd2821c9ce admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:52.018999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1085/2164] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:51 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:52.020807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057f694-3be0-4d17-9fac-bbfd2821c9ce admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:52.033316 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057f694-3be0-4d17-9fac-bbfd2821c9ce admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:52.040390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec25ab-36d6-4968-97cc-a56857e4f109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:52.041166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec25ab-36d6-4968-97cc-a56857e4f109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.041675 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec25ab-36d6-4968-97cc-a56857e4f109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.042173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec25ab-36d6-4968-97cc-a56857e4f109 admin admin] PATH_INFO: `/v3/projects/5d40210e0c9043359ce8dc1bb09074c7/users/92bbc8d7c672410c94c8062120a1580f/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.058107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec25ab-36d6-4968-97cc-a56857e4f109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d40210e0c9043359ce8dc1bb09074c7, user_id=92bbc8d7c672410c94c8062120a1580f, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:52.061105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec25ab-36d6-4968-97cc-a56857e4f109 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:52.067004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6762169-aadc-43a1-88e6-65e73751308a service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:52.071132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fee70-1370-4760-9023-9abdfcc18579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:52.072040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fee70-1370-4760-9023-9abdfcc18579 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.072401 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fee70-1370-4760-9023-9abdfcc18579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.072683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fee70-1370-4760-9023-9abdfcc18579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.081106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1086/2165] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:52 2022] PUT /identity/v3/projects/5d40210e0c9043359ce8dc1bb09074c7/users/92bbc8d7c672410c94c8062120a1580f/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:52.082224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec25ab-36d6-4968-97cc-a56857e4f109 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:52.082630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fee70-1370-4760-9023-9abdfcc18579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:52.084109 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fee70-1370-4760-9023-9abdfcc18579 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:52.107130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1080/2166] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:37:52.171192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec25ab-36d6-4968-97cc-a56857e4f109 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:52.178321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77268ffb-958f-4f4d-9e3e-8c74e90642e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:52.179265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268ffb-958f-4f4d-9e3e-8c74e90642e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.179608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268ffb-958f-4f4d-9e3e-8c74e90642e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.180015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268ffb-958f-4f4d-9e3e-8c74e90642e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.181156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77268ffb-958f-4f4d-9e3e-8c74e90642e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:52.182982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77268ffb-958f-4f4d-9e3e-8c74e90642e8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:52.199538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1087/2167] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:52 2022] GET /identity/v3/roles => generated 1427 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:52.200584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fee70-1370-4760-9023-9abdfcc18579 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:52.212809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fee70-1370-4760-9023-9abdfcc18579 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:52.219066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646fd20-822d-4a6f-86d8-7ba3cf4c433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:52.219978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646fd20-822d-4a6f-86d8-7ba3cf4c433d admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.220265 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646fd20-822d-4a6f-86d8-7ba3cf4c433d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.220546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646fd20-822d-4a6f-86d8-7ba3cf4c433d admin admin] PATH_INFO: `/v3/projects/5d40210e0c9043359ce8dc1bb09074c7/users/92bbc8d7c672410c94c8062120a1580f/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.225043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646fd20-822d-4a6f-86d8-7ba3cf4c433d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d40210e0c9043359ce8dc1bb09074c7, user_id=92bbc8d7c672410c94c8062120a1580f, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:52.238176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646fd20-822d-4a6f-86d8-7ba3cf4c433d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:52.261698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1081/2168] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:52 2022] PUT /identity/v3/projects/5d40210e0c9043359ce8dc1bb09074c7/users/92bbc8d7c672410c94c8062120a1580f/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:52.272133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab814ea-a96d-4033-844b-9ca2020cd5a6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.272422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab814ea-a96d-4033-844b-9ca2020cd5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.272702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab814ea-a96d-4033-844b-9ca2020cd5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.339428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-aab814ea-a96d-4033-844b-9ca2020cd5a6 None None] MFA Rules not processed for user `92bbc8d7c672410c94c8062120a1580f`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:52.341202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aab814ea-a96d-4033-844b-9ca2020cd5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:52.373917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:52.436744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1088/2169] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:52.446700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646fd20-822d-4a6f-86d8-7ba3cf4c433d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:52.449658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1646fd20-822d-4a6f-86d8-7ba3cf4c433d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:52.488374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab814ea-a96d-4033-844b-9ca2020cd5a6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:52.490466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aab814ea-a96d-4033-844b-9ca2020cd5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:52.536584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646fd20-822d-4a6f-86d8-7ba3cf4c433d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:52.541721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2393d-039a-4895-8354-766f80b8c4df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:52.542354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2393d-039a-4895-8354-766f80b8c4df service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.542641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2393d-039a-4895-8354-766f80b8c4df service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.542918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2393d-039a-4895-8354-766f80b8c4df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.555999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2393d-039a-4895-8354-766f80b8c4df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:52.558122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2393d-039a-4895-8354-766f80b8c4df service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:52.568794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1082/2170] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:52.587717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab814ea-a96d-4033-844b-9ca2020cd5a6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:52.593420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:52.594370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.594494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.594804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] PATH_INFO: `/v3/projects/3b47a5d24dc14c16a5b56a3fdaaf7c04` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.596704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b47a5d24dc14c16a5b56a3fdaaf7c04)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:52.598551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:52.666620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b47a5d24dc14c16a5b56a3fdaaf7c04', 'request_id': 'req-78bfc816-a638-4c17-abfb-28ed4c22cd0a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.679750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b47a5d24dc14c16a5b56a3fdaaf7c04', 'request_id': 'req-78bfc816-a638-4c17-abfb-28ed4c22cd0a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.695429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2393d-039a-4895-8354-766f80b8c4df service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:52.707912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2393d-039a-4895-8354-766f80b8c4df service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:52.709114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b47a5d24dc14c16a5b56a3fdaaf7c04', 'request_id': 'req-78bfc816-a638-4c17-abfb-28ed4c22cd0a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.722935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] The token cache is being invalidate because project 3b47a5d24dc14c16a5b56a3fdaaf7c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:52.723234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.723921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.729373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1089/2171] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:52 2022] DELETE /identity/v3/projects/3b47a5d24dc14c16a5b56a3fdaaf7c04 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:52.731214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:52.732584 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:52.803639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:52.805612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ceeb0-7034-4af8-be58-ac880a519b94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:52.806537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ceeb0-7034-4af8-be58-ac880a519b94 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.806892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ceeb0-7034-4af8-be58-ac880a519b94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.807506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ceeb0-7034-4af8-be58-ac880a519b94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.810064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3e7ceeb0-7034-4af8-be58-ac880a519b94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:52.824548 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfc816-a638-4c17-abfb-28ed4c22cd0a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:52.826000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:52.829777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:52.830425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:52.830708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:52.830981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] PATH_INFO: `/v3/users/45c4302907d8405bbec32629ce7bb6f0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:52.832260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c4302907d8405bbec32629ce7bb6f0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:52.834705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:52.886016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ceeb0-7034-4af8-be58-ac880a519b94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:52.887476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ceeb0-7034-4af8-be58-ac880a519b94 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:52.888832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c4302907d8405bbec32629ce7bb6f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.897555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1083/2172] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:52.903203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c4302907d8405bbec32629ce7bb6f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.911176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c4302907d8405bbec32629ce7bb6f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.921280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c4302907d8405bbec32629ce7bb6f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.922199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-722d2ee0-975b-4ba2-a5e2-020fcb628af3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c4302907d8405bbec32629ce7bb6f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:52.930674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1090/2173] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:52 2022] DELETE /identity/v3/users/45c4302907d8405bbec32629ce7bb6f0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:53.266574 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ceeb0-7034-4af8-be58-ac880a519b94 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:53.278228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ceeb0-7034-4af8-be58-ac880a519b94 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:53.338733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:53.339373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.339652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.339935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] PATH_INFO: `/v3/projects/7b02e862098a4377b3327e69a7e1eeaf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.341073 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b02e862098a4377b3327e69a7e1eeaf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:53.343006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:53.390258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b02e862098a4377b3327e69a7e1eeaf', 'request_id': 'req-98fd427f-bc31-4d92-ae75-312a024b3604'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:53.396407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b02e862098a4377b3327e69a7e1eeaf', 'request_id': 'req-98fd427f-bc31-4d92-ae75-312a024b3604'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:53.403098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b02e862098a4377b3327e69a7e1eeaf', 'request_id': 'req-98fd427f-bc31-4d92-ae75-312a024b3604'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:53.420063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] The token cache is being invalidate because project 7b02e862098a4377b3327e69a7e1eeaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:53.420354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:53.421012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98fd427f-bc31-4d92-ae75-312a024b3604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:53.422525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1084/2174] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:53 2022] DELETE /identity/v3/projects/7b02e862098a4377b3327e69a7e1eeaf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:53.429081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08268c1a-21d7-4e81-a392-d257347fde94 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.429366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08268c1a-21d7-4e81-a392-d257347fde94 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.429682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08268c1a-21d7-4e81-a392-d257347fde94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.450182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-08268c1a-21d7-4e81-a392-d257347fde94 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:53.451411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-08268c1a-21d7-4e81-a392-d257347fde94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:53.515209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1091/2175] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:53.526204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81cd5a-565d-4016-b97a-535a92712cc0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.526506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81cd5a-565d-4016-b97a-535a92712cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.526801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81cd5a-565d-4016-b97a-535a92712cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.547853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5d81cd5a-565d-4016-b97a-535a92712cc0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:53.549045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5d81cd5a-565d-4016-b97a-535a92712cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:53.557391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1085/2176] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:53.561705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08268c1a-21d7-4e81-a392-d257347fde94 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:53.571593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08268c1a-21d7-4e81-a392-d257347fde94 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:53.576154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235081fa-8b19-4df2-9cfa-fa3cdf267e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:53.576745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235081fa-8b19-4df2-9cfa-fa3cdf267e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.577026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235081fa-8b19-4df2-9cfa-fa3cdf267e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.577301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235081fa-8b19-4df2-9cfa-fa3cdf267e71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.577983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235081fa-8b19-4df2-9cfa-fa3cdf267e71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:53.578287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235081fa-8b19-4df2-9cfa-fa3cdf267e71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:53.579876 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235081fa-8b19-4df2-9cfa-fa3cdf267e71 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:53.592077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1092/2177] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:53.596680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81cd5a-565d-4016-b97a-535a92712cc0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:53.607432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81cd5a-565d-4016-b97a-535a92712cc0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:53.612193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bfa57-95b0-48fa-8144-03f4d0690f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:53.612787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bfa57-95b0-48fa-8144-03f4d0690f98 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.613077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bfa57-95b0-48fa-8144-03f4d0690f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.613337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bfa57-95b0-48fa-8144-03f4d0690f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.614029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bfa57-95b0-48fa-8144-03f4d0690f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:53.615918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bfa57-95b0-48fa-8144-03f4d0690f98 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:53.645683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1086/2178] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:53 2022] POST /identity/v3/projects => generated 374 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:53.650549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235081fa-8b19-4df2-9cfa-fa3cdf267e71 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:53.661294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235081fa-8b19-4df2-9cfa-fa3cdf267e71 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:53.728040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:53.744702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fc366-bbb1-44ad-ad82-2bedde6477c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:53.745338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fc366-bbb1-44ad-ad82-2bedde6477c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.745630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fc366-bbb1-44ad-ad82-2bedde6477c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.745919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fc366-bbb1-44ad-ad82-2bedde6477c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.746700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fc366-bbb1-44ad-ad82-2bedde6477c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:53.748510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fc366-bbb1-44ad-ad82-2bedde6477c8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:53.777545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1093/2179] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:53 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:53.782133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bfa57-95b0-48fa-8144-03f4d0690f98 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:53.798160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bfa57-95b0-48fa-8144-03f4d0690f98 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:53.798476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f948b9-4b0b-4779-98d4-c2140ef5ba0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:53.799057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f948b9-4b0b-4779-98d4-c2140ef5ba0c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.799335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f948b9-4b0b-4779-98d4-c2140ef5ba0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.799606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f948b9-4b0b-4779-98d4-c2140ef5ba0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.800257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f948b9-4b0b-4779-98d4-c2140ef5ba0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:53.801877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f948b9-4b0b-4779-98d4-c2140ef5ba0c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:53.812562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1087/2180] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:53 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:53.816646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fc366-bbb1-44ad-ad82-2bedde6477c8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:53.827981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fc366-bbb1-44ad-ad82-2bedde6477c8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:53.835760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8cba3-f281-46d3-a6ae-64fc2a721fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:53.837310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8cba3-f281-46d3-a6ae-64fc2a721fde admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.837846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8cba3-f281-46d3-a6ae-64fc2a721fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.838226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8cba3-f281-46d3-a6ae-64fc2a721fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.838878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8cba3-f281-46d3-a6ae-64fc2a721fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:53.840522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8cba3-f281-46d3-a6ae-64fc2a721fde admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:53.852535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1094/2181] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:53 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:53.856842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f948b9-4b0b-4779-98d4-c2140ef5ba0c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:53.863474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:53.868234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f948b9-4b0b-4779-98d4-c2140ef5ba0c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:53.875449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163b178-468c-4c12-8ec7-b03e4cd1bac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:53.876346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163b178-468c-4c12-8ec7-b03e4cd1bac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.876827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163b178-468c-4c12-8ec7-b03e4cd1bac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.877270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163b178-468c-4c12-8ec7-b03e4cd1bac0 admin admin] PATH_INFO: `/v3/projects/fc598dee680e440d9a71b62773207c13/users/ca0924947eb64c04ae1e643881ad767d/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.895688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163b178-468c-4c12-8ec7-b03e4cd1bac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc598dee680e440d9a71b62773207c13, user_id=ca0924947eb64c04ae1e643881ad767d, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:53.898414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163b178-468c-4c12-8ec7-b03e4cd1bac0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:53.913047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1088/2182] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:53 2022] PUT /identity/v3/projects/fc598dee680e440d9a71b62773207c13/users/ca0924947eb64c04ae1e643881ad767d/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:53.924925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9e4d6-63e6-4486-9db6-de8080752635 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:53.925207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9e4d6-63e6-4486-9db6-de8080752635 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:53.925488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9e4d6-63e6-4486-9db6-de8080752635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:53.955046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-37d9e4d6-63e6-4486-9db6-de8080752635 None None] MFA Rules not processed for user `ca0924947eb64c04ae1e643881ad767d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:53.956268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-37d9e4d6-63e6-4486-9db6-de8080752635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:54.030381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1095/2183] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:54.031903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163b178-468c-4c12-8ec7-b03e4cd1bac0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:54.099142 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163b178-468c-4c12-8ec7-b03e4cd1bac0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:54.104626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fadf69-bc75-45f5-bbbe-50d239c96321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:54.105231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fadf69-bc75-45f5-bbbe-50d239c96321 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:54.105511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fadf69-bc75-45f5-bbbe-50d239c96321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:54.105834 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fadf69-bc75-45f5-bbbe-50d239c96321 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:54.106483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fadf69-bc75-45f5-bbbe-50d239c96321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:54.108383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fadf69-bc75-45f5-bbbe-50d239c96321 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:54.137226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1089/2184] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:54 2022] POST /identity/v3/projects => generated 376 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:54.142080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9e4d6-63e6-4486-9db6-de8080752635 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:54.155157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9e4d6-63e6-4486-9db6-de8080752635 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:54.219316 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a788ab-b27c-4c93-9937-5268d2d9908b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:54.219944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a788ab-b27c-4c93-9937-5268d2d9908b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:54.220222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a788ab-b27c-4c93-9937-5268d2d9908b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:54.220491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a788ab-b27c-4c93-9937-5268d2d9908b admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:54.221132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a788ab-b27c-4c93-9937-5268d2d9908b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:54.223125 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a788ab-b27c-4c93-9937-5268d2d9908b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:54.246301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1096/2185] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:54 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:54.250346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fadf69-bc75-45f5-bbbe-50d239c96321 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:54.262272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fadf69-bc75-45f5-bbbe-50d239c96321 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:54.268069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2ddb2-44de-4a8a-98da-c55ffb37120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:54.268677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2ddb2-44de-4a8a-98da-c55ffb37120f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:54.268961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2ddb2-44de-4a8a-98da-c55ffb37120f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:54.269232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2ddb2-44de-4a8a-98da-c55ffb37120f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:54.269962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2ddb2-44de-4a8a-98da-c55ffb37120f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:54.271551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2ddb2-44de-4a8a-98da-c55ffb37120f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:54.284566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1090/2186] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:54 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:54.288915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a788ab-b27c-4c93-9937-5268d2d9908b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:54.299451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a788ab-b27c-4c93-9937-5268d2d9908b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:54.304269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a8aa7-52c3-413d-8ec3-c28e8594f6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:54.304926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a8aa7-52c3-413d-8ec3-c28e8594f6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:54.305282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a8aa7-52c3-413d-8ec3-c28e8594f6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:54.305726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a8aa7-52c3-413d-8ec3-c28e8594f6fd admin admin] PATH_INFO: `/v3/projects/ff46f00746af487b9e123468e723c2a0/users/e68f3b2d6f6b4acf86ff2b39f4c5b2ea/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:54.323913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a8aa7-52c3-413d-8ec3-c28e8594f6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff46f00746af487b9e123468e723c2a0, user_id=e68f3b2d6f6b4acf86ff2b39f4c5b2ea, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:54.326687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a8aa7-52c3-413d-8ec3-c28e8594f6fd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:54.342479 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1097/2187] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:54 2022] PUT /identity/v3/projects/ff46f00746af487b9e123468e723c2a0/users/e68f3b2d6f6b4acf86ff2b39f4c5b2ea/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:54.346391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2ddb2-44de-4a8a-98da-c55ffb37120f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:54.357015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2ddb2-44de-4a8a-98da-c55ffb37120f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:54.429688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ff98d-341a-4bb6-a1c4-de99b6f4ea36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:54.431263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ff98d-341a-4bb6-a1c4-de99b6f4ea36 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:54.431555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ff98d-341a-4bb6-a1c4-de99b6f4ea36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:54.431830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ff98d-341a-4bb6-a1c4-de99b6f4ea36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:54.432480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ff98d-341a-4bb6-a1c4-de99b6f4ea36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:54.434112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ff98d-341a-4bb6-a1c4-de99b6f4ea36 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:54.450972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1091/2188] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:54 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:54.452328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a8aa7-52c3-413d-8ec3-c28e8594f6fd admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:54.467597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a8aa7-52c3-413d-8ec3-c28e8594f6fd admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:54.472852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0ec26-db11-4798-9357-9aeb9fff270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:54.473774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0ec26-db11-4798-9357-9aeb9fff270f admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:54.485630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0ec26-db11-4798-9357-9aeb9fff270f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:54.485630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0ec26-db11-4798-9357-9aeb9fff270f admin admin] PATH_INFO: `/v3/projects/ff46f00746af487b9e123468e723c2a0/users/e68f3b2d6f6b4acf86ff2b39f4c5b2ea/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:54.485630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0ec26-db11-4798-9357-9aeb9fff270f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff46f00746af487b9e123468e723c2a0, user_id=e68f3b2d6f6b4acf86ff2b39f4c5b2ea, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:54.485630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0ec26-db11-4798-9357-9aeb9fff270f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:54.493491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1098/2189] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:54 2022] PUT /identity/v3/projects/ff46f00746af487b9e123468e723c2a0/users/e68f3b2d6f6b4acf86ff2b39f4c5b2ea/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:54.504197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb793af3-2187-4378-8339-7457960767af None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:54.504475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb793af3-2187-4378-8339-7457960767af None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:54.504813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb793af3-2187-4378-8339-7457960767af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:54.539610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bb793af3-2187-4378-8339-7457960767af None None] MFA Rules not processed for user `e68f3b2d6f6b4acf86ff2b39f4c5b2ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:54.540967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bb793af3-2187-4378-8339-7457960767af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:54.588091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:54.623909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1092/2190] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:54.636210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0ec26-db11-4798-9357-9aeb9fff270f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:54.638359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aae0ec26-db11-4798-9357-9aeb9fff270f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:54.667164 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:54.729153 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0ec26-db11-4798-9357-9aeb9fff270f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:54.733950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47f71b-1058-4d59-9f26-dd4c6cac697d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:54.734571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47f71b-1058-4d59-9f26-dd4c6cac697d service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:54.734843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47f71b-1058-4d59-9f26-dd4c6cac697d service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:54.735106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47f71b-1058-4d59-9f26-dd4c6cac697d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:54.826552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47f71b-1058-4d59-9f26-dd4c6cac697d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:54.827903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47f71b-1058-4d59-9f26-dd4c6cac697d service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:54.840405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1099/2191] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:55.563929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3ab7d-eb4f-4b2d-8141-3129074dbeae None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:55.564617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3ab7d-eb4f-4b2d-8141-3129074dbeae None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:55.564617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3ab7d-eb4f-4b2d-8141-3129074dbeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:55.585563 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-06c3ab7d-eb4f-4b2d-8141-3129074dbeae None None] MFA Rules not processed for user `75de510214004bbeab187d65bf178735`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:55.586877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-06c3ab7d-eb4f-4b2d-8141-3129074dbeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:55.663137 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1093/2192] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:56.670502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47f71b-1058-4d59-9f26-dd4c6cac697d service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:56.672924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5e47f71b-1058-4d59-9f26-dd4c6cac697d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:56.768264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47f71b-1058-4d59-9f26-dd4c6cac697d service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:56.773481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:56.774692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:56.774973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:56.775250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] PATH_INFO: `/v3/users/09ceb5de5c9a4540aa3d79bf72ee15f0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:56.776407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ceb5de5c9a4540aa3d79bf72ee15f0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:56.778369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:56.830023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ceb5de5c9a4540aa3d79bf72ee15f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:56.843881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ceb5de5c9a4540aa3d79bf72ee15f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:56.848841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:56.853336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ceb5de5c9a4540aa3d79bf72ee15f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:56.861979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ceb5de5c9a4540aa3d79bf72ee15f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:56.862508 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ceb5de5c9a4540aa3d79bf72ee15f0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:56.869510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1100/2193] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:56 2022] DELETE /identity/v3/users/09ceb5de5c9a4540aa3d79bf72ee15f0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:56.873970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6864f26-77a3-447d-8ace-988bebeaca2b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:56.874411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6864f26-77a3-447d-8ace-988bebeaca2b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:56.874672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6864f26-77a3-447d-8ace-988bebeaca2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:56.894182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f6864f26-77a3-447d-8ace-988bebeaca2b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:56.895379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f6864f26-77a3-447d-8ace-988bebeaca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:56.963407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1094/2194] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:56.972965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:56.974424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:57.044435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0efde-00cf-4dd1-a876-be66df5ca636 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:57.049143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ecbe1-0eb6-4a38-a335-02c758ebf5dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:57.049876 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ecbe1-0eb6-4a38-a335-02c758ebf5dc service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:57.050241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ecbe1-0eb6-4a38-a335-02c758ebf5dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:57.050693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ecbe1-0eb6-4a38-a335-02c758ebf5dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:57.060361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ecbe1-0eb6-4a38-a335-02c758ebf5dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:57.061975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ecbe1-0eb6-4a38-a335-02c758ebf5dc service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:57.071473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1101/2195] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:57.352488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6864f26-77a3-447d-8ace-988bebeaca2b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:57.354087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f6864f26-77a3-447d-8ace-988bebeaca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:57.386559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6864f26-77a3-447d-8ace-988bebeaca2b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:57.392494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:57.393134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:57.393408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:57.393713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] PATH_INFO: `/v3/projects/cec39bcf21204fa2a2bf2f005e23b9df` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:57.394848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec39bcf21204fa2a2bf2f005e23b9df)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:57.396787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:57.443457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec39bcf21204fa2a2bf2f005e23b9df', 'request_id': 'req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.450332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec39bcf21204fa2a2bf2f005e23b9df', 'request_id': 'req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.456085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec39bcf21204fa2a2bf2f005e23b9df', 'request_id': 'req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.469084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] The token cache is being invalidate because project cec39bcf21204fa2a2bf2f005e23b9df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:57.469303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.469942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.471457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1095/2196] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:57 2022] DELETE /identity/v3/projects/cec39bcf21204fa2a2bf2f005e23b9df => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:57.475387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ecbe1-0eb6-4a38-a335-02c758ebf5dc service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:57.485566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ecbe1-0eb6-4a38-a335-02c758ebf5dc service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:57.545162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:57.545821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:57.546098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:57.546368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] PATH_INFO: `/v3/users/92bbc8d7c672410c94c8062120a1580f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:57.547512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92bbc8d7c672410c94c8062120a1580f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:57.549474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:57.597098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92bbc8d7c672410c94c8062120a1580f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.609011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92bbc8d7c672410c94c8062120a1580f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.615709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92bbc8d7c672410c94c8062120a1580f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.623822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92bbc8d7c672410c94c8062120a1580f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.624427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b754bbe3-c867-473e-99fa-722fbb6a2bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92bbc8d7c672410c94c8062120a1580f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:57.630347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1102/2197] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:37:57 2022] DELETE /identity/v3/users/92bbc8d7c672410c94c8062120a1580f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:57.985494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:57.987050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:57.996435 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:58.011102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f04aa5-fd56-40a7-8b85-68a6d2ebd00f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:58.075045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:58.075683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.076034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.076277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] PATH_INFO: `/v3/projects/5d40210e0c9043359ce8dc1bb09074c7` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.077499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d40210e0c9043359ce8dc1bb09074c7)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:58.079494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:58.128694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d40210e0c9043359ce8dc1bb09074c7', 'request_id': 'req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:58.135850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d40210e0c9043359ce8dc1bb09074c7', 'request_id': 'req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:58.141656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d40210e0c9043359ce8dc1bb09074c7', 'request_id': 'req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:58.155631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] The token cache is being invalidate because project 5d40210e0c9043359ce8dc1bb09074c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:37:58.155920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:58.156546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f9bf4ab6-a13d-4ef0-9157-d802f0e4ad68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:37:58.158074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1096/2198] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:37:57 2022] DELETE /identity/v3/projects/5d40210e0c9043359ce8dc1bb09074c7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:58.163261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7de3d-c0bd-4aba-a5c9-815b9847fc75 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.163720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7de3d-c0bd-4aba-a5c9-815b9847fc75 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.163839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7de3d-c0bd-4aba-a5c9-815b9847fc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.183496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-73c7de3d-c0bd-4aba-a5c9-815b9847fc75 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:58.184706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-73c7de3d-c0bd-4aba-a5c9-815b9847fc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:58.188953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:58.247365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1103/2199] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:58.256160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f981b5b-cbee-42fd-a3cf-f17eb9463132 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.256448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f981b5b-cbee-42fd-a3cf-f17eb9463132 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.256934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f981b5b-cbee-42fd-a3cf-f17eb9463132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.277591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6f981b5b-cbee-42fd-a3cf-f17eb9463132 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:58.278820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6f981b5b-cbee-42fd-a3cf-f17eb9463132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:58.286994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1097/2200] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:58.291489 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7de3d-c0bd-4aba-a5c9-815b9847fc75 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:58.301283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7de3d-c0bd-4aba-a5c9-815b9847fc75 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:58.306494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a3f67-9560-4f64-b608-269346ab8c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:58.307090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a3f67-9560-4f64-b608-269346ab8c16 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.307373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a3f67-9560-4f64-b608-269346ab8c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.307647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a3f67-9560-4f64-b608-269346ab8c16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.308296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a3f67-9560-4f64-b608-269346ab8c16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:37:58.308600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a3f67-9560-4f64-b608-269346ab8c16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:58.310246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a3f67-9560-4f64-b608-269346ab8c16 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:58.330924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1104/2201] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:37:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:37:58.332150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f981b5b-cbee-42fd-a3cf-f17eb9463132 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:58.343619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f981b5b-cbee-42fd-a3cf-f17eb9463132 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:58.349101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fb0ee-0d75-45af-adc7-fca34b43faf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:58.349974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fb0ee-0d75-45af-adc7-fca34b43faf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.350410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fb0ee-0d75-45af-adc7-fca34b43faf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.350853 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fb0ee-0d75-45af-adc7-fca34b43faf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.351640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fb0ee-0d75-45af-adc7-fca34b43faf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:58.363497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fb0ee-0d75-45af-adc7-fca34b43faf6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:58.414303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1098/2202] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:58 2022] POST /identity/v3/projects => generated 402 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:58.419374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a3f67-9560-4f64-b608-269346ab8c16 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:58.431469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a3f67-9560-4f64-b608-269346ab8c16 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:58.505441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa867cf-6179-4826-9645-6466f60d0307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:58.505994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa867cf-6179-4826-9645-6466f60d0307 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.506281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa867cf-6179-4826-9645-6466f60d0307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.506547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa867cf-6179-4826-9645-6466f60d0307 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.507185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa867cf-6179-4826-9645-6466f60d0307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:58.509775 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa867cf-6179-4826-9645-6466f60d0307 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:58.535032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1105/2203] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:58 2022] POST /identity/v3/users => generated 302 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:58.539582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fb0ee-0d75-45af-adc7-fca34b43faf6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:58.551809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fb0ee-0d75-45af-adc7-fca34b43faf6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:58.563597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25894d-92b6-4beb-97dc-986f2394e7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:58.564342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25894d-92b6-4beb-97dc-986f2394e7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.564630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25894d-92b6-4beb-97dc-986f2394e7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.566040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25894d-92b6-4beb-97dc-986f2394e7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.566040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25894d-92b6-4beb-97dc-986f2394e7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:58.567196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25894d-92b6-4beb-97dc-986f2394e7a9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:58.580486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1099/2204] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:58 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:37:58.585559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa867cf-6179-4826-9645-6466f60d0307 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:58.601090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa867cf-6179-4826-9645-6466f60d0307 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:58.607878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714e3fc-3e16-4b7d-b3d0-a57541bae275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:58.608500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714e3fc-3e16-4b7d-b3d0-a57541bae275 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.608781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714e3fc-3e16-4b7d-b3d0-a57541bae275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.609058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714e3fc-3e16-4b7d-b3d0-a57541bae275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.609749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714e3fc-3e16-4b7d-b3d0-a57541bae275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:58.611365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714e3fc-3e16-4b7d-b3d0-a57541bae275 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:58.625023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1106/2205] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:58 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:58.629389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25894d-92b6-4beb-97dc-986f2394e7a9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:58.642220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25894d-92b6-4beb-97dc-986f2394e7a9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:58.651972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c60b6-6fc3-48b6-b39b-ed20e44153d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:58.652709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c60b6-6fc3-48b6-b39b-ed20e44153d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.652955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c60b6-6fc3-48b6-b39b-ed20e44153d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.653236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c60b6-6fc3-48b6-b39b-ed20e44153d9 admin admin] PATH_INFO: `/v3/projects/72762a3e75d848938760bf07600868ab/users/85e8b3a1fa7d47478252f10c8b2b3e06/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.672397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c60b6-6fc3-48b6-b39b-ed20e44153d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72762a3e75d848938760bf07600868ab, user_id=85e8b3a1fa7d47478252f10c8b2b3e06, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:58.675219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c60b6-6fc3-48b6-b39b-ed20e44153d9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:58.696250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1100/2206] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:58 2022] PUT /identity/v3/projects/72762a3e75d848938760bf07600868ab/users/85e8b3a1fa7d47478252f10c8b2b3e06/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:58.702417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721fe17-468b-4ed5-99f1-b13366b7091a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.702751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721fe17-468b-4ed5-99f1-b13366b7091a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.702981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721fe17-468b-4ed5-99f1-b13366b7091a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.735924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c721fe17-468b-4ed5-99f1-b13366b7091a None None] MFA Rules not processed for user `85e8b3a1fa7d47478252f10c8b2b3e06`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:58.737244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c721fe17-468b-4ed5-99f1-b13366b7091a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:58.817622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1107/2207] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:58 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:37:58.822981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c60b6-6fc3-48b6-b39b-ed20e44153d9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:58.861664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:58.902447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c60b6-6fc3-48b6-b39b-ed20e44153d9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:58.907616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd28b36-0585-4172-9967-5ad7ddb82623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:58.908238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd28b36-0585-4172-9967-5ad7ddb82623 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:58.908517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd28b36-0585-4172-9967-5ad7ddb82623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:58.908985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd28b36-0585-4172-9967-5ad7ddb82623 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:58.909755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd28b36-0585-4172-9967-5ad7ddb82623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:58.912015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd28b36-0585-4172-9967-5ad7ddb82623 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:58.941783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1101/2208] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:37:58 2022] POST /identity/v3/projects => generated 402 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:58.946847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721fe17-468b-4ed5-99f1-b13366b7091a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:58.953568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:58.958556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721fe17-468b-4ed5-99f1-b13366b7091a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:59.029431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb597fc-538f-4272-a8e6-6590e7d4536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:59.029981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb597fc-538f-4272-a8e6-6590e7d4536f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:59.030259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb597fc-538f-4272-a8e6-6590e7d4536f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:59.030528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb597fc-538f-4272-a8e6-6590e7d4536f admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:59.031179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb597fc-538f-4272-a8e6-6590e7d4536f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:59.033094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb597fc-538f-4272-a8e6-6590e7d4536f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:59.062346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1108/2209] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:37:58 2022] POST /identity/v3/users => generated 308 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:37:59.064164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd28b36-0585-4172-9967-5ad7ddb82623 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:59.076237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd28b36-0585-4172-9967-5ad7ddb82623 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:59.081914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ac288-4724-4825-aee5-3c7c62dda45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:59.082530 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ac288-4724-4825-aee5-3c7c62dda45d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:59.082809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ac288-4724-4825-aee5-3c7c62dda45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:59.083078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ac288-4724-4825-aee5-3c7c62dda45d admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:59.083720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ac288-4724-4825-aee5-3c7c62dda45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:59.085330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ac288-4724-4825-aee5-3c7c62dda45d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:59.098825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1102/2210] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:59 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:59.103230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb597fc-538f-4272-a8e6-6590e7d4536f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:59.119987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb597fc-538f-4272-a8e6-6590e7d4536f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:59.125921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c5e31-8932-461f-a7bb-87f5fa6e1013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:59.126604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c5e31-8932-461f-a7bb-87f5fa6e1013 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:59.126897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c5e31-8932-461f-a7bb-87f5fa6e1013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:59.127249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c5e31-8932-461f-a7bb-87f5fa6e1013 admin admin] PATH_INFO: `/v3/projects/59098310490e45c1adb87575216ee3ef/users/84d29631e72d4420950ac5defd23462f/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:59.148614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c5e31-8932-461f-a7bb-87f5fa6e1013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59098310490e45c1adb87575216ee3ef, user_id=84d29631e72d4420950ac5defd23462f, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:59.151848 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c5e31-8932-461f-a7bb-87f5fa6e1013 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:59.177711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1109/2211] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:59 2022] PUT /identity/v3/projects/59098310490e45c1adb87575216ee3ef/users/84d29631e72d4420950ac5defd23462f/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:37:59.179337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ac288-4724-4825-aee5-3c7c62dda45d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:59.196552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ac288-4724-4825-aee5-3c7c62dda45d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:59.256945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93443484-babc-48a0-a888-c409cc5befba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:59.257556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93443484-babc-48a0-a888-c409cc5befba admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:59.257866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93443484-babc-48a0-a888-c409cc5befba admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:59.258802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93443484-babc-48a0-a888-c409cc5befba admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:59.259684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93443484-babc-48a0-a888-c409cc5befba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:59.261190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93443484-babc-48a0-a888-c409cc5befba admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:59.271710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1103/2212] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:37:59 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:37:59.275839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c5e31-8932-461f-a7bb-87f5fa6e1013 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:59.286458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c5e31-8932-461f-a7bb-87f5fa6e1013 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:59.294091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a99390-372e-478d-882f-682b890ac357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:59.294854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a99390-372e-478d-882f-682b890ac357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:59.295205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a99390-372e-478d-882f-682b890ac357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:59.295574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a99390-372e-478d-882f-682b890ac357 admin admin] PATH_INFO: `/v3/projects/59098310490e45c1adb87575216ee3ef/users/84d29631e72d4420950ac5defd23462f/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:59.297597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a99390-372e-478d-882f-682b890ac357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59098310490e45c1adb87575216ee3ef, user_id=84d29631e72d4420950ac5defd23462f, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:59.300949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a99390-372e-478d-882f-682b890ac357 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:59.313465 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1110/2213] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:37:59 2022] PUT /identity/v3/projects/59098310490e45c1adb87575216ee3ef/users/84d29631e72d4420950ac5defd23462f/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:37:59.324587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdc604-5a51-40e4-a212-e568ba5a68e0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:59.324917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdc604-5a51-40e4-a212-e568ba5a68e0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:59.325172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdc604-5a51-40e4-a212-e568ba5a68e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:59.363610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c7bdc604-5a51-40e4-a212-e568ba5a68e0 None None] MFA Rules not processed for user `84d29631e72d4420950ac5defd23462f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:37:59.365519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c7bdc604-5a51-40e4-a212-e568ba5a68e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:59.461628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1104/2214] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:37:59 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:37:59.471522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a99390-372e-478d-882f-682b890ac357 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:59.474458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c9a99390-372e-478d-882f-682b890ac357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:37:59.544999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a99390-372e-478d-882f-682b890ac357 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:59.549259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26281c97-c859-4dbd-a268-0780ea8122e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:59.549903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26281c97-c859-4dbd-a268-0780ea8122e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:59.550186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26281c97-c859-4dbd-a268-0780ea8122e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:59.550457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26281c97-c859-4dbd-a268-0780ea8122e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:59.560822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26281c97-c859-4dbd-a268-0780ea8122e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:59.562296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26281c97-c859-4dbd-a268-0780ea8122e8 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:59.574744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1111/2215] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:59 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:37:59.662505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdc604-5a51-40e4-a212-e568ba5a68e0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:37:59.673538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdc604-5a51-40e4-a212-e568ba5a68e0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:37:59.678222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9b8f4-34cd-47aa-b584-44a55c4f2573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:37:59.678822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9b8f4-34cd-47aa-b584-44a55c4f2573 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:37:59.679108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9b8f4-34cd-47aa-b584-44a55c4f2573 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:37:59.679378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9b8f4-34cd-47aa-b584-44a55c4f2573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:37:59.706456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:37:59.747368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9b8f4-34cd-47aa-b584-44a55c4f2573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:37:59.748812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9b8f4-34cd-47aa-b584-44a55c4f2573 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:37:59.758614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1105/2216] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:37:59 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:01.756383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26281c97-c859-4dbd-a268-0780ea8122e8 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:01.758147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-26281c97-c859-4dbd-a268-0780ea8122e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:01.840102 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26281c97-c859-4dbd-a268-0780ea8122e8 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:01.846385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:01.847118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:01.847405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:01.847794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] PATH_INFO: `/v3/users/ca0924947eb64c04ae1e643881ad767d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:01.849018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0924947eb64c04ae1e643881ad767d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:01.851587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:01.854477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:01.940429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0924947eb64c04ae1e643881ad767d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:01.966398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0924947eb64c04ae1e643881ad767d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:01.976866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0924947eb64c04ae1e643881ad767d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:01.994156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0924947eb64c04ae1e643881ad767d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:01.995039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0924947eb64c04ae1e643881ad767d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:02.003754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1112/2217] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:01 2022] DELETE /identity/v3/users/ca0924947eb64c04ae1e643881ad767d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:02.008799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22386da1-ecd7-4545-94d9-400038f8e704 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:02.009307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22386da1-ecd7-4545-94d9-400038f8e704 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:02.009307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22386da1-ecd7-4545-94d9-400038f8e704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:02.047163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-22386da1-ecd7-4545-94d9-400038f8e704 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:02.048757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-22386da1-ecd7-4545-94d9-400038f8e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:02.071516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:02.073223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:02.116252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1106/2218] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:02.127172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22386da1-ecd7-4545-94d9-400038f8e704 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:02.129384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-22386da1-ecd7-4545-94d9-400038f8e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:02.151380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cbb74-4de8-427a-a5aa-33bf458e599a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:02.156584 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:02.157325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:02.157705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:02.158052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] PATH_INFO: `/v3/users/daff4a5dca684cc080f5cf0140202eff` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:02.159398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=daff4a5dca684cc080f5cf0140202eff)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:02.161435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:02.204621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22386da1-ecd7-4545-94d9-400038f8e704 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:02.211319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0230ae-93a1-43e3-927a-e949a586977f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:02.212573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0230ae-93a1-43e3-927a-e949a586977f service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:02.212921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0230ae-93a1-43e3-927a-e949a586977f service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:02.213929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0230ae-93a1-43e3-927a-e949a586977f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:02.237783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0230ae-93a1-43e3-927a-e949a586977f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:02.239840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0230ae-93a1-43e3-927a-e949a586977f service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:02.248531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daff4a5dca684cc080f5cf0140202eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:02.255182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1107/2219] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:02.266024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daff4a5dca684cc080f5cf0140202eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:02.274847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daff4a5dca684cc080f5cf0140202eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:02.286355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daff4a5dca684cc080f5cf0140202eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:02.287611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daff4a5dca684cc080f5cf0140202eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:02.291128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:02.295696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1113/2220] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:02 2022] DELETE /identity/v3/users/daff4a5dca684cc080f5cf0140202eff => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:38:02.300089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb9bfa-61f6-49e4-ac78-d2850fa5517b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:02.300404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb9bfa-61f6-49e4-ac78-d2850fa5517b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:02.301692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb9bfa-61f6-49e4-ac78-d2850fa5517b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:02.327458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-18bb9bfa-61f6-49e4-ac78-d2850fa5517b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:02.329004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-18bb9bfa-61f6-49e4-ac78-d2850fa5517b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:02.399942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1108/2221] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:02.417228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:02.419063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:02.522550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326e12-0c79-42b2-ad7a-c67de1defe8e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:02.528571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fc364-bfa9-45f5-926a-0e19c06777b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:02.529383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fc364-bfa9-45f5-926a-0e19c06777b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:02.530954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fc364-bfa9-45f5-926a-0e19c06777b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:02.531855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fc364-bfa9-45f5-926a-0e19c06777b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:02.558299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fc364-bfa9-45f5-926a-0e19c06777b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:02.559760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fc364-bfa9-45f5-926a-0e19c06777b3 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:02.571307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1114/2222] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:38:03.015846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb9bfa-61f6-49e4-ac78-d2850fa5517b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:03.017962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-18bb9bfa-61f6-49e4-ac78-d2850fa5517b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:03.039342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb9bfa-61f6-49e4-ac78-d2850fa5517b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:03.087215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:03.090108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:03.095331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:03.095893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] PATH_INFO: `/v3/projects/fc598dee680e440d9a71b62773207c13` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:03.102282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc598dee680e440d9a71b62773207c13)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:03.112338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:03.123082 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:03.247076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc598dee680e440d9a71b62773207c13', 'request_id': 'req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.257238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc598dee680e440d9a71b62773207c13', 'request_id': 'req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.280658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc598dee680e440d9a71b62773207c13', 'request_id': 'req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.312680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] The token cache is being invalidate because project fc598dee680e440d9a71b62773207c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:03.313601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.313857 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.316354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1109/2223] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:03 2022] DELETE /identity/v3/projects/fc598dee680e440d9a71b62773207c13 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:03.320470 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fc364-bfa9-45f5-926a-0e19c06777b3 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:03.334374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fc364-bfa9-45f5-926a-0e19c06777b3 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:03.358329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:03.362453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:03.408804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:03.410357 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:03.412001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:03.413903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] PATH_INFO: `/v3/users/e68f3b2d6f6b4acf86ff2b39f4c5b2ea` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:03.422603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68f3b2d6f6b4acf86ff2b39f4c5b2ea)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:03.426977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:03.496169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bb8e8-0fcb-4670-bdd5-d71e2d16de40 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:03.502673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:03.503533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:03.503832 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:03.504172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] PATH_INFO: `/v3/projects/1f3c298dfe9b49a084f1cfd602ffd609` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:03.505648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f3c298dfe9b49a084f1cfd602ffd609)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:03.508052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:03.539626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68f3b2d6f6b4acf86ff2b39f4c5b2ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.554063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68f3b2d6f6b4acf86ff2b39f4c5b2ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.565184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68f3b2d6f6b4acf86ff2b39f4c5b2ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.577415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68f3b2d6f6b4acf86ff2b39f4c5b2ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.578035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68f3b2d6f6b4acf86ff2b39f4c5b2ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.589223 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f3c298dfe9b49a084f1cfd602ffd609', 'request_id': 'req-7ee49222-ed6f-414b-9347-8abd1d4b8aea'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.590444 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1115/2224] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:03 2022] DELETE /identity/v3/users/e68f3b2d6f6b4acf86ff2b39f4c5b2ea => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:03.610781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f3c298dfe9b49a084f1cfd602ffd609', 'request_id': 'req-7ee49222-ed6f-414b-9347-8abd1d4b8aea'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.621224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f3c298dfe9b49a084f1cfd602ffd609', 'request_id': 'req-7ee49222-ed6f-414b-9347-8abd1d4b8aea'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.646768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] The token cache is being invalidate because project 1f3c298dfe9b49a084f1cfd602ffd609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:03.647454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.648542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.651884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1110/2225] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:03 2022] DELETE /identity/v3/projects/1f3c298dfe9b49a084f1cfd602ffd609 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:03.656405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:03.658661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:03.705915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:03.747749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8aeaa-24de-491a-8c45-8e5faa9e9759 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:03.753738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:03.754421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:03.754705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:03.754978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] PATH_INFO: `/v3/users/6775b87e75b5484a9b6290127da0cc0b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:03.756152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6775b87e75b5484a9b6290127da0cc0b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:03.758325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:03.810426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6775b87e75b5484a9b6290127da0cc0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.824774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6775b87e75b5484a9b6290127da0cc0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.834452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6775b87e75b5484a9b6290127da0cc0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.842421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6775b87e75b5484a9b6290127da0cc0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.843207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6775b87e75b5484a9b6290127da0cc0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:03.853620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1116/2226] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:03 2022] DELETE /identity/v3/users/6775b87e75b5484a9b6290127da0cc0b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:04.158029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:04.159783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.196937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:04.198792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.215325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:04.301334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee49222-ed6f-414b-9347-8abd1d4b8aea admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:04.337346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7d3b6-c921-4211-beb5-2d26f529bd95 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:04.349504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:04.350460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.350847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.351805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] PATH_INFO: `/v3/users/85e8b3a1fa7d47478252f10c8b2b3e06` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.354304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e8b3a1fa7d47478252f10c8b2b3e06)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:04.356460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:04.434936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:04.435675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.436058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.436611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] PATH_INFO: `/v3/projects/ff46f00746af487b9e123468e723c2a0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.438001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff46f00746af487b9e123468e723c2a0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:04.440986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:04.464279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:04.472926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e8b3a1fa7d47478252f10c8b2b3e06'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.497771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e8b3a1fa7d47478252f10c8b2b3e06'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.507459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e8b3a1fa7d47478252f10c8b2b3e06'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.518749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e8b3a1fa7d47478252f10c8b2b3e06'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.519390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e8b3a1fa7d47478252f10c8b2b3e06'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.526854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1117/2227] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:04 2022] DELETE /identity/v3/users/85e8b3a1fa7d47478252f10c8b2b3e06 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:38:04.527432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:04.528997 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.546906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff46f00746af487b9e123468e723c2a0', 'request_id': 'req-661665ed-917f-47e3-8ec8-c711d512258f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.562063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff46f00746af487b9e123468e723c2a0', 'request_id': 'req-661665ed-917f-47e3-8ec8-c711d512258f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.562063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff46f00746af487b9e123468e723c2a0', 'request_id': 'req-661665ed-917f-47e3-8ec8-c711d512258f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.578858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] The token cache is being invalidate because project ff46f00746af487b9e123468e723c2a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:04.579257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.579969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-661665ed-917f-47e3-8ec8-c711d512258f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.581545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1111/2228] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:04 2022] DELETE /identity/v3/projects/ff46f00746af487b9e123468e723c2a0 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:04.582856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afa53a-676e-4b61-8706-a8cccf664d79 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.583270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afa53a-676e-4b61-8706-a8cccf664d79 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.583650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afa53a-676e-4b61-8706-a8cccf664d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.604054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb0cd3-c046-48f7-a6c7-a80ca62eb939 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:04.604938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-81afa53a-676e-4b61-8706-a8cccf664d79 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:04.606183 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-81afa53a-676e-4b61-8706-a8cccf664d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.608812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:04.609446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.609761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.610031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] PATH_INFO: `/v3/projects/14a834aa47984964bbb82a8d017b60c5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.611116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a834aa47984964bbb82a8d017b60c5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:04.613046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:04.613919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1112/2229] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:04.617295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb96e4-ca77-452a-b983-2b4b5224a6ae None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.617295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb96e4-ca77-452a-b983-2b4b5224a6ae None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.617295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb96e4-ca77-452a-b983-2b4b5224a6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.638352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2fdb96e4-ca77-452a-b983-2b4b5224a6ae None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:04.639687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2fdb96e4-ca77-452a-b983-2b4b5224a6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.657373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1113/2230] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:04.657373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb96e4-ca77-452a-b983-2b4b5224a6ae None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:04.657373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2fdb96e4-ca77-452a-b983-2b4b5224a6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.670652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a834aa47984964bbb82a8d017b60c5', 'request_id': 'req-61b2f8ba-d938-4c2c-8583-dffb280a741a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.679116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a834aa47984964bbb82a8d017b60c5', 'request_id': 'req-61b2f8ba-d938-4c2c-8583-dffb280a741a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.685517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a834aa47984964bbb82a8d017b60c5', 'request_id': 'req-61b2f8ba-d938-4c2c-8583-dffb280a741a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.697206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] The token cache is being invalidate because project 14a834aa47984964bbb82a8d017b60c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:04.697910 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.698491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61b2f8ba-d938-4c2c-8583-dffb280a741a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:04.699991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1118/2231] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:04 2022] DELETE /identity/v3/projects/14a834aa47984964bbb82a8d017b60c5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:04.701266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435c7b5-4bba-43e4-8132-a6fb95ead594 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.701666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435c7b5-4bba-43e4-8132-a6fb95ead594 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.702078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435c7b5-4bba-43e4-8132-a6fb95ead594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.713602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:04.725101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3435c7b5-4bba-43e4-8132-a6fb95ead594 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:04.725290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb96e4-ca77-452a-b983-2b4b5224a6ae None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:04.726328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3435c7b5-4bba-43e4-8132-a6fb95ead594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.776689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:04.793848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad58db3-eb07-4c85-a592-7e5ec6ef23e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:04.794470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad58db3-eb07-4c85-a592-7e5ec6ef23e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.794750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad58db3-eb07-4c85-a592-7e5ec6ef23e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.795021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad58db3-eb07-4c85-a592-7e5ec6ef23e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.796794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0ad58db3-eb07-4c85-a592-7e5ec6ef23e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.801410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1119/2232] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:04.801410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9bccb-e762-4741-b3f3-cb932b9f61ca None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.801410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9bccb-e762-4741-b3f3-cb932b9f61ca None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.801410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9bccb-e762-4741-b3f3-cb932b9f61ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.812562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad58db3-eb07-4c85-a592-7e5ec6ef23e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:04.814043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad58db3-eb07-4c85-a592-7e5ec6ef23e4 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:04.822066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0ec9bccb-e762-4741-b3f3-cb932b9f61ca None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:04.823386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0ec9bccb-e762-4741-b3f3-cb932b9f61ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.827122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1114/2233] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:04.828017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad58db3-eb07-4c85-a592-7e5ec6ef23e4 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:04.832054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1120/2234] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:04.839780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad58db3-eb07-4c85-a592-7e5ec6ef23e4 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:04.841496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb0a7f-d627-4a48-8759-791f9d361957 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.841816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb0a7f-d627-4a48-8759-791f9d361957 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.842118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb0a7f-d627-4a48-8759-791f9d361957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.844821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dd7b9-12d9-4ce9-aec1-ab46c73d76c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:04.845437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dd7b9-12d9-4ce9-aec1-ab46c73d76c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.845732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dd7b9-12d9-4ce9-aec1-ab46c73d76c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.846013 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dd7b9-12d9-4ce9-aec1-ab46c73d76c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.846664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56dd7b9-12d9-4ce9-aec1-ab46c73d76c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:04.846972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56dd7b9-12d9-4ce9-aec1-ab46c73d76c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:04.848571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56dd7b9-12d9-4ce9-aec1-ab46c73d76c3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:04.865813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1115/2235] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:38:04.867135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dd7b9-12d9-4ce9-aec1-ab46c73d76c3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:04.867713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-98fb0a7f-d627-4a48-8759-791f9d361957 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:04.869103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-98fb0a7f-d627-4a48-8759-791f9d361957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:04.878158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dd7b9-12d9-4ce9-aec1-ab46c73d76c3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:04.878339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1121/2236] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:04.882179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb0a7f-d627-4a48-8759-791f9d361957 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:04.883000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd00300-b96d-482d-a62f-78f20734421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:04.883614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd00300-b96d-482d-a62f-78f20734421c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.884593 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd00300-b96d-482d-a62f-78f20734421c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.884957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd00300-b96d-482d-a62f-78f20734421c admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.885617 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd00300-b96d-482d-a62f-78f20734421c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:04.887620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd00300-b96d-482d-a62f-78f20734421c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:04.892792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb0a7f-d627-4a48-8759-791f9d361957 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:04.897608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a85653-017d-461f-b146-b0344679b185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:04.898248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a85653-017d-461f-b146-b0344679b185 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:04.898538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a85653-017d-461f-b146-b0344679b185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:04.898810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a85653-017d-461f-b146-b0344679b185 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:04.900061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a85653-017d-461f-b146-b0344679b185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:04.900371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a85653-017d-461f-b146-b0344679b185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:04.902000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a85653-017d-461f-b146-b0344679b185 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:04.922755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1122/2237] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:38:04.924080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a85653-017d-461f-b146-b0344679b185 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:04.933940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1116/2238] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:04 2022] POST /identity/v3/projects => generated 376 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:38:04.936199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd00300-b96d-482d-a62f-78f20734421c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:04.938512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a85653-017d-461f-b146-b0344679b185 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.010523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b37ec-b046-4366-8ab6-96895df00f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.011704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b37ec-b046-4366-8ab6-96895df00f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.011957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b37ec-b046-4366-8ab6-96895df00f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.012337 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b37ec-b046-4366-8ab6-96895df00f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.013172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b37ec-b046-4366-8ab6-96895df00f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.014372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd00300-b96d-482d-a62f-78f20734421c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.015644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b37ec-b046-4366-8ab6-96895df00f55 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.019757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6c2b4-8e24-4329-88bc-e359262e604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.021406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6c2b4-8e24-4329-88bc-e359262e604d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.021651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6c2b4-8e24-4329-88bc-e359262e604d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.022081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6c2b4-8e24-4329-88bc-e359262e604d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.022821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6c2b4-8e24-4329-88bc-e359262e604d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.025408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6c2b4-8e24-4329-88bc-e359262e604d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.052192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1117/2239] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:04 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:38:05.057335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6c2b4-8e24-4329-88bc-e359262e604d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.068902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1123/2240] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:04 2022] POST /identity/v3/projects => generated 366 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:05.069110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6c2b4-8e24-4329-88bc-e359262e604d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.070046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b37ec-b046-4366-8ab6-96895df00f55 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.140522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7a180-3df0-42b9-ae0d-d4f684d58873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.141137 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7a180-3df0-42b9-ae0d-d4f684d58873 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.141417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7a180-3df0-42b9-ae0d-d4f684d58873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.141720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7a180-3df0-42b9-ae0d-d4f684d58873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.142261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b37ec-b046-4366-8ab6-96895df00f55 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.142508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7a180-3df0-42b9-ae0d-d4f684d58873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.144119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7a180-3df0-42b9-ae0d-d4f684d58873 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.145670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:05.148182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d2775-4a8e-43ec-9fa0-2b764bab01c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.148806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d2775-4a8e-43ec-9fa0-2b764bab01c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.149087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d2775-4a8e-43ec-9fa0-2b764bab01c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.149437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d2775-4a8e-43ec-9fa0-2b764bab01c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.150273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d2775-4a8e-43ec-9fa0-2b764bab01c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.152553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d2775-4a8e-43ec-9fa0-2b764bab01c7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.168027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1118/2241] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:05 2022] GET /identity/v3/roles => generated 1427 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:38:05.169666 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7a180-3df0-42b9-ae0d-d4f684d58873 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.180060 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1124/2242] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:05 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:05.183436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d2775-4a8e-43ec-9fa0-2b764bab01c7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.184256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7a180-3df0-42b9-ae0d-d4f684d58873 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.185151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2f0d2775-4a8e-43ec-9fa0-2b764bab01c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:05.189473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae76288-c6fb-4ff6-8290-e3a2b89bfb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.190052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae76288-c6fb-4ff6-8290-e3a2b89bfb4a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.190335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae76288-c6fb-4ff6-8290-e3a2b89bfb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.190614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae76288-c6fb-4ff6-8290-e3a2b89bfb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.191266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae76288-c6fb-4ff6-8290-e3a2b89bfb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.193183 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae76288-c6fb-4ff6-8290-e3a2b89bfb4a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.198401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d2775-4a8e-43ec-9fa0-2b764bab01c7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.203680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.204696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.204963 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.205232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] PATH_INFO: `/v3/projects/72762a3e75d848938760bf07600868ab` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.206437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72762a3e75d848938760bf07600868ab)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.208341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.214915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1119/2243] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:05 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:05.216698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae76288-c6fb-4ff6-8290-e3a2b89bfb4a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.227532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae76288-c6fb-4ff6-8290-e3a2b89bfb4a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.234300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff359e07-e074-473f-bacd-811395c2cb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.235132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff359e07-e074-473f-bacd-811395c2cb01 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.235487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff359e07-e074-473f-bacd-811395c2cb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.235900 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff359e07-e074-473f-bacd-811395c2cb01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.236820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff359e07-e074-473f-bacd-811395c2cb01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.238445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff359e07-e074-473f-bacd-811395c2cb01 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.253138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1120/2244] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:05 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:05.253138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff359e07-e074-473f-bacd-811395c2cb01 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.254857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:05.263923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff359e07-e074-473f-bacd-811395c2cb01 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.270639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72762a3e75d848938760bf07600868ab', 'request_id': 'req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.279635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72762a3e75d848938760bf07600868ab', 'request_id': 'req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.289031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72762a3e75d848938760bf07600868ab', 'request_id': 'req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.301326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] The token cache is being invalidate because project 72762a3e75d848938760bf07600868ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:05.301649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.302333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.303879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1125/2245] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:05 2022] DELETE /identity/v3/projects/72762a3e75d848938760bf07600868ab => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:05.304655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.306535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:05.328526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7d6f0-4f89-44a6-893f-f3ce8b6a96ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.329273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d6f0-4f89-44a6-893f-f3ce8b6a96ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.329553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d6f0-4f89-44a6-893f-f3ce8b6a96ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.329888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d6f0-4f89-44a6-893f-f3ce8b6a96ff admin admin] PATH_INFO: `/v3/projects/8643f179145e4e1aba7994ada9821a30/users/c2e0a450471043678b080515896d0f50/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.344860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7d6f0-4f89-44a6-893f-f3ce8b6a96ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8643f179145e4e1aba7994ada9821a30, user_id=c2e0a450471043678b080515896d0f50, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.348373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7d6f0-4f89-44a6-893f-f3ce8b6a96ff admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.372324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1121/2246] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:05 2022] PUT /identity/v3/projects/8643f179145e4e1aba7994ada9821a30/users/c2e0a450471043678b080515896d0f50/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:05.372324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7d6f0-4f89-44a6-893f-f3ce8b6a96ff admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.372774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-97a7d6f0-4f89-44a6-893f-f3ce8b6a96ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:05.379637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ce2a8-d13b-4a13-8a13-078cf89c84a4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.441674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7d6f0-4f89-44a6-893f-f3ce8b6a96ff admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.447879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.448538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.448812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.449086 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] PATH_INFO: `/v3/users/84d29631e72d4420950ac5defd23462f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.450257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d29631e72d4420950ac5defd23462f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.450901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a49fb-6e1d-48e9-afad-99130f56a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.451654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a49fb-6e1d-48e9-afad-99130f56a537 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.452120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a49fb-6e1d-48e9-afad-99130f56a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.452218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.452405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a49fb-6e1d-48e9-afad-99130f56a537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.453059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a49fb-6e1d-48e9-afad-99130f56a537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.455904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a49fb-6e1d-48e9-afad-99130f56a537 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.467619 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1126/2247] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:05 2022] GET /identity/v3/roles => generated 1427 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:05.468968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9eed6-8f31-494f-8cbf-b79260092845 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.470006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9eed6-8f31-494f-8cbf-b79260092845 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.470006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9eed6-8f31-494f-8cbf-b79260092845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.501301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-82a9eed6-8f31-494f-8cbf-b79260092845 None None] MFA Rules not processed for user `c2e0a450471043678b080515896d0f50`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:05.503814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-82a9eed6-8f31-494f-8cbf-b79260092845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:05.526062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d29631e72d4420950ac5defd23462f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.536089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d29631e72d4420950ac5defd23462f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.537815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d29631e72d4420950ac5defd23462f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.546266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:05.563408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d29631e72d4420950ac5defd23462f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.585371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:05.587898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d29631e72d4420950ac5defd23462f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:05.596606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1122/2248] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:05 2022] DELETE /identity/v3/users/84d29631e72d4420950ac5defd23462f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:05.601321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.603983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:05.607570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1127/2249] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:05.616953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9eed6-8f31-494f-8cbf-b79260092845 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.619099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-82a9eed6-8f31-494f-8cbf-b79260092845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:05.701085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9eed6-8f31-494f-8cbf-b79260092845 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.736235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca504726-d136-4921-97c5-b108ddbf4320 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.742599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c6e6d-3d2d-4dec-8280-7612dbe916df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.744077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c6e6d-3d2d-4dec-8280-7612dbe916df admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.744077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c6e6d-3d2d-4dec-8280-7612dbe916df admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.744664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c6e6d-3d2d-4dec-8280-7612dbe916df admin admin] PATH_INFO: `/v3/projects/450ad9198bed4ecfad1f0bb708cd1726/users/82c211533f924c58af1b43aba5c9594a/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.763580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c6e6d-3d2d-4dec-8280-7612dbe916df admin admin] RBAC: Authorizing `identity:create_grant(project_id=450ad9198bed4ecfad1f0bb708cd1726, user_id=82c211533f924c58af1b43aba5c9594a, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.767382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c6e6d-3d2d-4dec-8280-7612dbe916df admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.781637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204c37c-b96c-467b-8584-0a7ebd40fff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:05.782727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204c37c-b96c-467b-8584-0a7ebd40fff6 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.783144 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204c37c-b96c-467b-8584-0a7ebd40fff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.783392 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1123/2250] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:05 2022] PUT /identity/v3/projects/450ad9198bed4ecfad1f0bb708cd1726/users/82c211533f924c58af1b43aba5c9594a/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:05.786075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204c37c-b96c-467b-8584-0a7ebd40fff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.788047 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7204c37c-b96c-467b-8584-0a7ebd40fff6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:05.794676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c52737-24bd-4be0-bde4-3c2c6b40a298 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:05.794958 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c52737-24bd-4be0-bde4-3c2c6b40a298 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:05.795231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c52737-24bd-4be0-bde4-3c2c6b40a298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:05.826770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-60c52737-24bd-4be0-bde4-3c2c6b40a298 None None] MFA Rules not processed for user `82c211533f924c58af1b43aba5c9594a`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:05.828057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-60c52737-24bd-4be0-bde4-3c2c6b40a298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:05.881538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204c37c-b96c-467b-8584-0a7ebd40fff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:05.883199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204c37c-b96c-467b-8584-0a7ebd40fff6 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:05.895369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1128/2251] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:05.901874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1124/2252] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:05.906407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204c37c-b96c-467b-8584-0a7ebd40fff6 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.918851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204c37c-b96c-467b-8584-0a7ebd40fff6 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:05.963788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c52737-24bd-4be0-bde4-3c2c6b40a298 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:05.965263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-60c52737-24bd-4be0-bde4-3c2c6b40a298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:06.014438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16a031-9e8e-4805-8e52-ec9c16d18d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.015178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16a031-9e8e-4805-8e52-ec9c16d18d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.015348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16a031-9e8e-4805-8e52-ec9c16d18d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.017239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16a031-9e8e-4805-8e52-ec9c16d18d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.018917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16a031-9e8e-4805-8e52-ec9c16d18d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.024116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16a031-9e8e-4805-8e52-ec9c16d18d0d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.030584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:06.043213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:06.056398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c52737-24bd-4be0-bde4-3c2c6b40a298 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.062827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.063519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.063788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.064112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] PATH_INFO: `/v3/projects/59098310490e45c1adb87575216ee3ef` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.065294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=59098310490e45c1adb87575216ee3ef)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.067293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.072775 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1129/2253] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:05 2022] POST /identity/v3/projects => generated 366 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:06.077313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16a031-9e8e-4805-8e52-ec9c16d18d0d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.155574 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59098310490e45c1adb87575216ee3ef', 'request_id': 'req-468101dd-a027-4cc9-abd7-fae5501cfb2f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:06.155574 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59098310490e45c1adb87575216ee3ef', 'request_id': 'req-468101dd-a027-4cc9-abd7-fae5501cfb2f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:06.158833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59098310490e45c1adb87575216ee3ef', 'request_id': 'req-468101dd-a027-4cc9-abd7-fae5501cfb2f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:06.175989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] The token cache is being invalidate because project 59098310490e45c1adb87575216ee3ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:06.176623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:06.177154 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-468101dd-a027-4cc9-abd7-fae5501cfb2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:06.178937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1125/2254] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:05 2022] DELETE /identity/v3/projects/59098310490e45c1adb87575216ee3ef => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:06.179053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16a031-9e8e-4805-8e52-ec9c16d18d0d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.185128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41136c-5e6f-46c4-8ad1-273d55a1dbef None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.185407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41136c-5e6f-46c4-8ad1-273d55a1dbef None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.185734 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41136c-5e6f-46c4-8ad1-273d55a1dbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.207242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2d41136c-5e6f-46c4-8ad1-273d55a1dbef None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:06.208504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2d41136c-5e6f-46c4-8ad1-273d55a1dbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:06.257699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da1b3f-5a6e-416c-be28-c1ce2984e732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.259163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da1b3f-5a6e-416c-be28-c1ce2984e732 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.259163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da1b3f-5a6e-416c-be28-c1ce2984e732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.259756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da1b3f-5a6e-416c-be28-c1ce2984e732 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.260402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da1b3f-5a6e-416c-be28-c1ce2984e732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.262845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da1b3f-5a6e-416c-be28-c1ce2984e732 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.293743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1126/2255] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:06.299315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a03bce-e04e-4764-9917-5faf4520e034 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.299707 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a03bce-e04e-4764-9917-5faf4520e034 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.299985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a03bce-e04e-4764-9917-5faf4520e034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.303359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1130/2256] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:06 2022] POST /identity/v3/users => generated 290 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:06.305666 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da1b3f-5a6e-416c-be28-c1ce2984e732 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.307701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a0da1b3f-5a6e-416c-be28-c1ce2984e732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:06.321759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c8a03bce-e04e-4764-9917-5faf4520e034 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:06.322614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da1b3f-5a6e-416c-be28-c1ce2984e732 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.323294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c8a03bce-e04e-4764-9917-5faf4520e034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:06.328410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c79d30-a882-4fdc-895d-d82c919a98be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.329152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79d30-a882-4fdc-895d-d82c919a98be admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.329534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79d30-a882-4fdc-895d-d82c919a98be admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.329951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79d30-a882-4fdc-895d-d82c919a98be admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.330780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c79d30-a882-4fdc-895d-d82c919a98be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.332710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c79d30-a882-4fdc-895d-d82c919a98be admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.332820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1127/2257] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:06.337598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a03bce-e04e-4764-9917-5faf4520e034 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.345630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1131/2258] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:06 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:06.349810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c79d30-a882-4fdc-895d-d82c919a98be admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.352976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a03bce-e04e-4764-9917-5faf4520e034 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.358787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245627c-ca25-40e6-9f81-1771d3092907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.359403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245627c-ca25-40e6-9f81-1771d3092907 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.359677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245627c-ca25-40e6-9f81-1771d3092907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.359942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245627c-ca25-40e6-9f81-1771d3092907 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.360458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c79d30-a882-4fdc-895d-d82c919a98be admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.360591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245627c-ca25-40e6-9f81-1771d3092907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:06.360889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245627c-ca25-40e6-9f81-1771d3092907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.362510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245627c-ca25-40e6-9f81-1771d3092907 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.365717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980b61b-a918-4703-a542-c0b3f140dd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.366371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980b61b-a918-4703-a542-c0b3f140dd8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.366652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980b61b-a918-4703-a542-c0b3f140dd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.366933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980b61b-a918-4703-a542-c0b3f140dd8d admin admin] PATH_INFO: `/v3/projects/27b5312feab04631a82c2e8445b48a01/users/fb06c1fc686f4b23875cac579b7beaa6/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.374576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1128/2259] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:38:06.379337 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245627c-ca25-40e6-9f81-1771d3092907 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.383160 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980b61b-a918-4703-a542-c0b3f140dd8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b5312feab04631a82c2e8445b48a01, user_id=fb06c1fc686f4b23875cac579b7beaa6, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.386454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980b61b-a918-4703-a542-c0b3f140dd8d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.391033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245627c-ca25-40e6-9f81-1771d3092907 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.396819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f7357-4bd2-4f47-9cb9-f6ab3faabf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.397363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7357-4bd2-4f47-9cb9-f6ab3faabf33 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.397662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7357-4bd2-4f47-9cb9-f6ab3faabf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.397934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7357-4bd2-4f47-9cb9-f6ab3faabf33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.398555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f7357-4bd2-4f47-9cb9-f6ab3faabf33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.404154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1132/2260] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:06 2022] PUT /identity/v3/projects/27b5312feab04631a82c2e8445b48a01/users/fb06c1fc686f4b23875cac579b7beaa6/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:38:06.404294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f7357-4bd2-4f47-9cb9-f6ab3faabf33 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.405340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980b61b-a918-4703-a542-c0b3f140dd8d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.434625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1129/2261] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:06 2022] POST /identity/v3/projects => generated 388 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:06.435979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f7357-4bd2-4f47-9cb9-f6ab3faabf33 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.461151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:06.474320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980b61b-a918-4703-a542-c0b3f140dd8d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.478302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:06.503788 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f7357-4bd2-4f47-9cb9-f6ab3faabf33 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.508659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb182e0-8b9d-42a4-9f2d-88e62c97c3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.509933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb182e0-8b9d-42a4-9f2d-88e62c97c3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.510222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb182e0-8b9d-42a4-9f2d-88e62c97c3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.510492 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb182e0-8b9d-42a4-9f2d-88e62c97c3b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.511161 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb182e0-8b9d-42a4-9f2d-88e62c97c3b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.513100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb182e0-8b9d-42a4-9f2d-88e62c97c3b7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.534970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c1f2a-24c6-4629-85bd-d291aef43de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.535579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c1f2a-24c6-4629-85bd-d291aef43de8 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.539191 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c1f2a-24c6-4629-85bd-d291aef43de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.539318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1130/2262] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:06 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:06.539438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c1f2a-24c6-4629-85bd-d291aef43de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.539438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c1f2a-24c6-4629-85bd-d291aef43de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.539438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c1f2a-24c6-4629-85bd-d291aef43de8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.540623 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb182e0-8b9d-42a4-9f2d-88e62c97c3b7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.550044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1133/2263] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:06 2022] GET /identity/v3/roles => generated 1427 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:38:06.550955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb182e0-8b9d-42a4-9f2d-88e62c97c3b7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.553637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c1f2a-24c6-4629-85bd-d291aef43de8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.556463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa5fe1-f2b4-40e8-b549-b2ed43aa6ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.557190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa5fe1-f2b4-40e8-b549-b2ed43aa6ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.557490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa5fe1-f2b4-40e8-b549-b2ed43aa6ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.557795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa5fe1-f2b4-40e8-b549-b2ed43aa6ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.558444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa5fe1-f2b4-40e8-b549-b2ed43aa6ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.560041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa5fe1-f2b4-40e8-b549-b2ed43aa6ff3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.564504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c1f2a-24c6-4629-85bd-d291aef43de8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.573278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1131/2264] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:06 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:38:06.573419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4005f11-2805-42ef-954b-5c6a40ad2d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.573419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4005f11-2805-42ef-954b-5c6a40ad2d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.573419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4005f11-2805-42ef-954b-5c6a40ad2d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.573419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4005f11-2805-42ef-954b-5c6a40ad2d71 admin admin] PATH_INFO: `/v3/projects/27b5312feab04631a82c2e8445b48a01/users/fb06c1fc686f4b23875cac579b7beaa6/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.575187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa5fe1-f2b4-40e8-b549-b2ed43aa6ff3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.575299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4005f11-2805-42ef-954b-5c6a40ad2d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b5312feab04631a82c2e8445b48a01, user_id=fb06c1fc686f4b23875cac579b7beaa6, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.577964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4005f11-2805-42ef-954b-5c6a40ad2d71 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.586070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa5fe1-f2b4-40e8-b549-b2ed43aa6ff3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.590037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1134/2265] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:06 2022] PUT /identity/v3/projects/27b5312feab04631a82c2e8445b48a01/users/fb06c1fc686f4b23875cac579b7beaa6/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:38:06.607431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659526b-b2ce-4454-adbc-c405c0b66d13 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.607944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659526b-b2ce-4454-adbc-c405c0b66d13 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.608443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659526b-b2ce-4454-adbc-c405c0b66d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.661311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e659526b-b2ce-4454-adbc-c405c0b66d13 None None] MFA Rules not processed for user `fb06c1fc686f4b23875cac579b7beaa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:06.663008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e659526b-b2ce-4454-adbc-c405c0b66d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:06.669484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0fc6c-a633-42c1-b68c-de67c518af98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.670221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0fc6c-a633-42c1-b68c-de67c518af98 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.670497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0fc6c-a633-42c1-b68c-de67c518af98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.670764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0fc6c-a633-42c1-b68c-de67c518af98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.671405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0fc6c-a633-42c1-b68c-de67c518af98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.673548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0fc6c-a633-42c1-b68c-de67c518af98 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.685021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1132/2266] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:06 2022] GET /identity/v3/roles => generated 1427 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:06.688978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0fc6c-a633-42c1-b68c-de67c518af98 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.699957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0fc6c-a633-42c1-b68c-de67c518af98 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.706070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1a48b-e58a-4a9f-9da6-6182a86dbaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.706070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1a48b-e58a-4a9f-9da6-6182a86dbaac admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.706320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1a48b-e58a-4a9f-9da6-6182a86dbaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.706530 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1a48b-e58a-4a9f-9da6-6182a86dbaac admin admin] PATH_INFO: `/v3/projects/2cafdfc397a64769a9c775198630734e/users/a7d6ea89451a46bf8e117b601bf53943/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.725965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1a48b-e58a-4a9f-9da6-6182a86dbaac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cafdfc397a64769a9c775198630734e, user_id=a7d6ea89451a46bf8e117b601bf53943, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.730125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1a48b-e58a-4a9f-9da6-6182a86dbaac admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.763226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1133/2267] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:06 2022] PUT /identity/v3/projects/2cafdfc397a64769a9c775198630734e/users/a7d6ea89451a46bf8e117b601bf53943/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:06.773550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac6518-1db0-425d-a550-df39e96aa905 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.773931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1135/2268] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:06.774878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac6518-1db0-425d-a550-df39e96aa905 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.775658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac6518-1db0-425d-a550-df39e96aa905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.781018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659526b-b2ce-4454-adbc-c405c0b66d13 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.782733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e659526b-b2ce-4454-adbc-c405c0b66d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:06.812168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-93ac6518-1db0-425d-a550-df39e96aa905 None None] MFA Rules not processed for user `a7d6ea89451a46bf8e117b601bf53943`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:06.813410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-93ac6518-1db0-425d-a550-df39e96aa905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:06.861663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659526b-b2ce-4454-adbc-c405c0b66d13 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.888823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:06.891336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1134/2269] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:06.897848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac6518-1db0-425d-a550-df39e96aa905 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:06.918889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:06.924002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2053e7-7175-49d0-bc04-5a6102b7c1e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.924745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2053e7-7175-49d0-bc04-5a6102b7c1e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.924971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2053e7-7175-49d0-bc04-5a6102b7c1e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.925239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2053e7-7175-49d0-bc04-5a6102b7c1e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.966522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac6518-1db0-425d-a550-df39e96aa905 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:06.971139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afcf22-7f15-4a03-87d3-e184cc6d077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:06.971744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afcf22-7f15-4a03-87d3-e184cc6d077e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:06.972021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afcf22-7f15-4a03-87d3-e184cc6d077e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:06.972412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afcf22-7f15-4a03-87d3-e184cc6d077e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:06.972920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afcf22-7f15-4a03-87d3-e184cc6d077e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.974837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afcf22-7f15-4a03-87d3-e184cc6d077e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:06.990855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2053e7-7175-49d0-bc04-5a6102b7c1e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:06.992256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2053e7-7175-49d0-bc04-5a6102b7c1e0 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:07.003416 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1135/2270] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:06 2022] POST /identity/v3/projects => generated 384 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:38:07.004067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1136/2271] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:07.007014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afcf22-7f15-4a03-87d3-e184cc6d077e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:07.080368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afcf22-7f15-4a03-87d3-e184cc6d077e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:07.085325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45856297-5b0c-497e-8a9e-7c87db3c0b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:07.085963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45856297-5b0c-497e-8a9e-7c87db3c0b37 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:07.086246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45856297-5b0c-497e-8a9e-7c87db3c0b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:07.086518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45856297-5b0c-497e-8a9e-7c87db3c0b37 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:07.088397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45856297-5b0c-497e-8a9e-7c87db3c0b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:07.090419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45856297-5b0c-497e-8a9e-7c87db3c0b37 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:07.091490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2053e7-7175-49d0-bc04-5a6102b7c1e0 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:07.101036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2053e7-7175-49d0-bc04-5a6102b7c1e0 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:07.112214 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1136/2272] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:07 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:07.115956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45856297-5b0c-497e-8a9e-7c87db3c0b37 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:07.125688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45856297-5b0c-497e-8a9e-7c87db3c0b37 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:07.131126 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fa15d-7669-4201-8591-c7329fb02e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:07.131715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fa15d-7669-4201-8591-c7329fb02e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:07.131992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fa15d-7669-4201-8591-c7329fb02e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:07.132324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fa15d-7669-4201-8591-c7329fb02e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:07.132973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fa15d-7669-4201-8591-c7329fb02e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:07.135284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fa15d-7669-4201-8591-c7329fb02e64 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:07.146043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1137/2273] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:07 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:38:07.150819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fa15d-7669-4201-8591-c7329fb02e64 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:07.155279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eda99-589c-443b-b118-c20fd7cdb660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:07.156156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eda99-589c-443b-b118-c20fd7cdb660 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:07.156522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eda99-589c-443b-b118-c20fd7cdb660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:07.156701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eda99-589c-443b-b118-c20fd7cdb660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:07.159719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c34eda99-589c-443b-b118-c20fd7cdb660 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:07.162705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fa15d-7669-4201-8591-c7329fb02e64 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:07.167863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33d9db-bd1f-437f-bdf6-6e1758c192be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:07.168539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33d9db-bd1f-437f-bdf6-6e1758c192be admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:07.168824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33d9db-bd1f-437f-bdf6-6e1758c192be admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:07.169100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33d9db-bd1f-437f-bdf6-6e1758c192be admin admin] PATH_INFO: `/v3/projects/d6eb66d0b8f54277888692b11e6f5c90/users/1846bb40ee794edd999e3e9f223e4e7b/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:07.185327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33d9db-bd1f-437f-bdf6-6e1758c192be admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6eb66d0b8f54277888692b11e6f5c90, user_id=1846bb40ee794edd999e3e9f223e4e7b, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:07.188067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33d9db-bd1f-437f-bdf6-6e1758c192be admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:07.205958 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1138/2274] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:07 2022] PUT /identity/v3/projects/d6eb66d0b8f54277888692b11e6f5c90/users/1846bb40ee794edd999e3e9f223e4e7b/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:38:07.209828 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33d9db-bd1f-437f-bdf6-6e1758c192be admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:07.229742 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eda99-589c-443b-b118-c20fd7cdb660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:07.231242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eda99-589c-443b-b118-c20fd7cdb660 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:07.240850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1137/2275] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:38:07.282970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:07.288151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33d9db-bd1f-437f-bdf6-6e1758c192be admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:07.297304 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e07ae-6826-4c87-94fa-9434a915922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:07.297772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e07ae-6826-4c87-94fa-9434a915922e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:07.298128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e07ae-6826-4c87-94fa-9434a915922e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:07.298518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e07ae-6826-4c87-94fa-9434a915922e admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:07.299252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e07ae-6826-4c87-94fa-9434a915922e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:07.300985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e07ae-6826-4c87-94fa-9434a915922e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:07.314698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1139/2276] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:07 2022] GET /identity/v3/roles => generated 1427 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:38:07.320620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eda99-589c-443b-b118-c20fd7cdb660 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:07.341752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eda99-589c-443b-b118-c20fd7cdb660 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:07.346713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbafcbf-b06a-4fec-b2d9-d90f14d5df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:07.347390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbafcbf-b06a-4fec-b2d9-d90f14d5df63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:07.347795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbafcbf-b06a-4fec-b2d9-d90f14d5df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:07.348083 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbafcbf-b06a-4fec-b2d9-d90f14d5df63 admin admin] PATH_INFO: `/v3/projects/d6eb66d0b8f54277888692b11e6f5c90/users/1846bb40ee794edd999e3e9f223e4e7b/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:07.350077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbafcbf-b06a-4fec-b2d9-d90f14d5df63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6eb66d0b8f54277888692b11e6f5c90, user_id=1846bb40ee794edd999e3e9f223e4e7b, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:07.353702 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbafcbf-b06a-4fec-b2d9-d90f14d5df63 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:07.373695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1138/2277] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:07 2022] PUT /identity/v3/projects/d6eb66d0b8f54277888692b11e6f5c90/users/1846bb40ee794edd999e3e9f223e4e7b/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:07.384779 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693f709-03cb-45e7-90f6-fd7ca4106e48 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:07.385129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693f709-03cb-45e7-90f6-fd7ca4106e48 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:07.385327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693f709-03cb-45e7-90f6-fd7ca4106e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:07.415019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3693f709-03cb-45e7-90f6-fd7ca4106e48 None None] MFA Rules not processed for user `1846bb40ee794edd999e3e9f223e4e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:07.416822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3693f709-03cb-45e7-90f6-fd7ca4106e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:07.506817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1140/2278] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:07.518974 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbafcbf-b06a-4fec-b2d9-d90f14d5df63 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:07.522158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5fbafcbf-b06a-4fec-b2d9-d90f14d5df63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:07.541322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:07.600406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbafcbf-b06a-4fec-b2d9-d90f14d5df63 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:07.605203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b7cdb-0981-4011-ae36-62cdf0426606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:07.605842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b7cdb-0981-4011-ae36-62cdf0426606 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:07.606127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b7cdb-0981-4011-ae36-62cdf0426606 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:07.606398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b7cdb-0981-4011-ae36-62cdf0426606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:07.616929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b7cdb-0981-4011-ae36-62cdf0426606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:07.619155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b7cdb-0981-4011-ae36-62cdf0426606 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:07.628985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1139/2279] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:08.323182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693f709-03cb-45e7-90f6-fd7ca4106e48 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:08.405477 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693f709-03cb-45e7-90f6-fd7ca4106e48 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:08.413124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:08.413748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:08.414025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:08.414344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] PATH_INFO: `/v3/users/a7d6ea89451a46bf8e117b601bf53943` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:08.416538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d6ea89451a46bf8e117b601bf53943)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:08.418774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:08.454762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c62e4-4f47-4323-93ca-f12af953aeb5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:08.455344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c62e4-4f47-4323-93ca-f12af953aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:08.455784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c62e4-4f47-4323-93ca-f12af953aeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:08.481370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-094c62e4-4f47-4323-93ca-f12af953aeb5 None None] MFA Rules not processed for user `c2e0a450471043678b080515896d0f50`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:08.482705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-094c62e4-4f47-4323-93ca-f12af953aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:08.482833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d6ea89451a46bf8e117b601bf53943'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:08.501703 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d6ea89451a46bf8e117b601bf53943'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:08.502393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d6ea89451a46bf8e117b601bf53943'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:08.523372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d6ea89451a46bf8e117b601bf53943'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:08.532973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f794cc4c-c7f2-4cc7-ac03-ce078e14d09f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d6ea89451a46bf8e117b601bf53943'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:08.534941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:08.543390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1141/2280] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:08 2022] DELETE /identity/v3/users/a7d6ea89451a46bf8e117b601bf53943 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:38:08.550240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a171-0d74-46b2-88cb-c0db35b05f91 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:08.551464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a171-0d74-46b2-88cb-c0db35b05f91 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:08.551922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a171-0d74-46b2-88cb-c0db35b05f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:08.554049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1140/2281] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:08.582204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4d42a171-0d74-46b2-88cb-c0db35b05f91 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:08.583536 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4d42a171-0d74-46b2-88cb-c0db35b05f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:08.680956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1142/2282] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:08.690826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c62e4-4f47-4323-93ca-f12af953aeb5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:08.692383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-094c62e4-4f47-4323-93ca-f12af953aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:08.770825 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c62e4-4f47-4323-93ca-f12af953aeb5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:08.851017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ba1bd-f7bd-4825-b301-e14dd47261dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:08.851738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ba1bd-f7bd-4825-b301-e14dd47261dd service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:08.852015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ba1bd-f7bd-4825-b301-e14dd47261dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:08.853012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ba1bd-f7bd-4825-b301-e14dd47261dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:08.871160 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ba1bd-f7bd-4825-b301-e14dd47261dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:08.872689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ba1bd-f7bd-4825-b301-e14dd47261dd service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:08.883321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1141/2283] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:09.443185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42a171-0d74-46b2-88cb-c0db35b05f91 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:09.445306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4d42a171-0d74-46b2-88cb-c0db35b05f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:09.472150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42a171-0d74-46b2-88cb-c0db35b05f91 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:09.478230 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:09.478991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:09.479433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:09.479891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] PATH_INFO: `/v3/projects/2cafdfc397a64769a9c775198630734e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:09.481097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cafdfc397a64769a9c775198630734e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:09.483290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:09.548619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cafdfc397a64769a9c775198630734e', 'request_id': 'req-7c17894f-6c5c-4bc8-a4e4-2965f750830a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.555837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cafdfc397a64769a9c775198630734e', 'request_id': 'req-7c17894f-6c5c-4bc8-a4e4-2965f750830a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.566501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cafdfc397a64769a9c775198630734e', 'request_id': 'req-7c17894f-6c5c-4bc8-a4e4-2965f750830a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.582594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] The token cache is being invalidate because project 2cafdfc397a64769a9c775198630734e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:09.582978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.583800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.585567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1143/2284] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:09 2022] DELETE /identity/v3/projects/2cafdfc397a64769a9c775198630734e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:09.590436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ba1bd-f7bd-4825-b301-e14dd47261dd service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:09.600862 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ba1bd-f7bd-4825-b301-e14dd47261dd service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:09.626500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:09.709314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:09.710392 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:09.710930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:09.712289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] PATH_INFO: `/v3/users/1846bb40ee794edd999e3e9f223e4e7b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:09.714109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1846bb40ee794edd999e3e9f223e4e7b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:09.716669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:09.786023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1846bb40ee794edd999e3e9f223e4e7b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.801171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1846bb40ee794edd999e3e9f223e4e7b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.812591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1846bb40ee794edd999e3e9f223e4e7b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.825839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1846bb40ee794edd999e3e9f223e4e7b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.826391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3f53f59-178f-44c9-980e-4265328cd1c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1846bb40ee794edd999e3e9f223e4e7b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:09.832852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1142/2285] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:09 2022] DELETE /identity/v3/users/1846bb40ee794edd999e3e9f223e4e7b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:10.276985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:10.278563 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:10.294310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17894f-6c5c-4bc8-a4e4-2965f750830a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:10.403560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:10.404558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:10.405253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:10.405253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] PATH_INFO: `/v3/projects/d6eb66d0b8f54277888692b11e6f5c90` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:10.406988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6eb66d0b8f54277888692b11e6f5c90)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:10.409237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:10.444446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:10.477427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6eb66d0b8f54277888692b11e6f5c90', 'request_id': 'req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:10.490654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6eb66d0b8f54277888692b11e6f5c90', 'request_id': 'req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:10.499219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6eb66d0b8f54277888692b11e6f5c90', 'request_id': 'req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:10.521248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] The token cache is being invalidate because project d6eb66d0b8f54277888692b11e6f5c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:10.521547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:10.522481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5b8cd1a5-4d98-487d-8089-7bfa8e3c5c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:10.527595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1144/2286] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:10 2022] DELETE /identity/v3/projects/d6eb66d0b8f54277888692b11e6f5c90 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:10.533003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce340e51-0715-4549-b660-605bf90a3e8c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:10.533301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce340e51-0715-4549-b660-605bf90a3e8c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:10.533568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce340e51-0715-4549-b660-605bf90a3e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:10.557953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ce340e51-0715-4549-b660-605bf90a3e8c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:10.561740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ce340e51-0715-4549-b660-605bf90a3e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:10.656783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1143/2287] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:10.671018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059e89e-87d7-4b9f-973c-21773a0d3e4f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:10.671749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059e89e-87d7-4b9f-973c-21773a0d3e4f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:10.672224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059e89e-87d7-4b9f-973c-21773a0d3e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:10.699855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4059e89e-87d7-4b9f-973c-21773a0d3e4f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:10.701377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4059e89e-87d7-4b9f-973c-21773a0d3e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:10.710172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1145/2288] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:10.715376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce340e51-0715-4549-b660-605bf90a3e8c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:10.725329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce340e51-0715-4549-b660-605bf90a3e8c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:10.738132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eebfe7-4ff2-42be-8cc6-221e2b7e3c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:10.738760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eebfe7-4ff2-42be-8cc6-221e2b7e3c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:10.739036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eebfe7-4ff2-42be-8cc6-221e2b7e3c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:10.739308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eebfe7-4ff2-42be-8cc6-221e2b7e3c51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:10.739956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eebfe7-4ff2-42be-8cc6-221e2b7e3c51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:10.740424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eebfe7-4ff2-42be-8cc6-221e2b7e3c51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:10.742755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eebfe7-4ff2-42be-8cc6-221e2b7e3c51 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:10.760432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1144/2289] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:38:10.761549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059e89e-87d7-4b9f-973c-21773a0d3e4f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:10.772319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059e89e-87d7-4b9f-973c-21773a0d3e4f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:10.779047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac19fc-4b30-463c-82e8-f95683b03542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:10.779676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac19fc-4b30-463c-82e8-f95683b03542 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:10.780255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac19fc-4b30-463c-82e8-f95683b03542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:10.780538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac19fc-4b30-463c-82e8-f95683b03542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:10.782062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac19fc-4b30-463c-82e8-f95683b03542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:10.784060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac19fc-4b30-463c-82e8-f95683b03542 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:10.820275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1146/2290] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:10 2022] POST /identity/v3/projects => generated 386 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:10.825046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eebfe7-4ff2-42be-8cc6-221e2b7e3c51 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:10.839064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eebfe7-4ff2-42be-8cc6-221e2b7e3c51 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:10.876023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:10.931915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7cff9-5041-4508-b76d-36bb59199000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:10.932792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7cff9-5041-4508-b76d-36bb59199000 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:10.933300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7cff9-5041-4508-b76d-36bb59199000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:10.933920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7cff9-5041-4508-b76d-36bb59199000 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:10.934716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7cff9-5041-4508-b76d-36bb59199000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:10.937012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7cff9-5041-4508-b76d-36bb59199000 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:10.979906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1145/2291] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:10 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:10.984146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac19fc-4b30-463c-82e8-f95683b03542 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:10.996826 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac19fc-4b30-463c-82e8-f95683b03542 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:11.004331 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b5ecc-429f-49d3-bd4a-7adba430ecd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:11.005286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b5ecc-429f-49d3-bd4a-7adba430ecd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:11.005816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b5ecc-429f-49d3-bd4a-7adba430ecd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:11.006411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b5ecc-429f-49d3-bd4a-7adba430ecd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:11.007862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b5ecc-429f-49d3-bd4a-7adba430ecd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:11.011290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b5ecc-429f-49d3-bd4a-7adba430ecd8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:11.027963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1147/2292] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:10 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:11.035599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7cff9-5041-4508-b76d-36bb59199000 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:11.048358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7cff9-5041-4508-b76d-36bb59199000 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:11.053155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612a227-0206-4c4d-8275-bc002e780ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:11.053794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612a227-0206-4c4d-8275-bc002e780ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:11.054073 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612a227-0206-4c4d-8275-bc002e780ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:11.054343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612a227-0206-4c4d-8275-bc002e780ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:11.054983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612a227-0206-4c4d-8275-bc002e780ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:11.057231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612a227-0206-4c4d-8275-bc002e780ccf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:11.074957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1146/2293] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:11 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:11.076783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b5ecc-429f-49d3-bd4a-7adba430ecd8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:11.087409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b5ecc-429f-49d3-bd4a-7adba430ecd8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:11.092509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8377b-3396-4da1-b478-cc42983baa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:11.093497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8377b-3396-4da1-b478-cc42983baa78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:11.093935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8377b-3396-4da1-b478-cc42983baa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:11.094277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8377b-3396-4da1-b478-cc42983baa78 admin admin] PATH_INFO: `/v3/projects/4a05b1fd52434d32bbf2333c73775509/users/9282e3834a754f69960a60ff290d9bac/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:11.116517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8377b-3396-4da1-b478-cc42983baa78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a05b1fd52434d32bbf2333c73775509, user_id=9282e3834a754f69960a60ff290d9bac, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:11.120783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8377b-3396-4da1-b478-cc42983baa78 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:11.137955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1148/2294] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:11 2022] PUT /identity/v3/projects/4a05b1fd52434d32bbf2333c73775509/users/9282e3834a754f69960a60ff290d9bac/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:11.148755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d399a27-ac10-4084-aecb-40c92dd38369 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:11.149124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d399a27-ac10-4084-aecb-40c92dd38369 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:11.149397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d399a27-ac10-4084-aecb-40c92dd38369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:11.179976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2d399a27-ac10-4084-aecb-40c92dd38369 None None] MFA Rules not processed for user `9282e3834a754f69960a60ff290d9bac`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:11.182272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d399a27-ac10-4084-aecb-40c92dd38369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:11.264699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1147/2295] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:11 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:11.273203 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8377b-3396-4da1-b478-cc42983baa78 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:11.275613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-21b8377b-3396-4da1-b478-cc42983baa78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:11.355172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8377b-3396-4da1-b478-cc42983baa78 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:11.362665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8634a-15e4-4c7a-831e-9decdb2d01cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:11.363464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8634a-15e4-4c7a-831e-9decdb2d01cc service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:11.363894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8634a-15e4-4c7a-831e-9decdb2d01cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:11.364309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8634a-15e4-4c7a-831e-9decdb2d01cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:11.382075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8634a-15e4-4c7a-831e-9decdb2d01cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:11.383486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8634a-15e4-4c7a-831e-9decdb2d01cc service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:11.393644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1149/2296] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:11 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:12.558705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d399a27-ac10-4084-aecb-40c92dd38369 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:12.560301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d399a27-ac10-4084-aecb-40c92dd38369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:12.641086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d399a27-ac10-4084-aecb-40c92dd38369 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:12.646304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:12.647008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:12.647294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:12.647622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] PATH_INFO: `/v3/users/82c211533f924c58af1b43aba5c9594a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:12.648769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c211533f924c58af1b43aba5c9594a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:12.651190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:12.722659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:12.737700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c211533f924c58af1b43aba5c9594a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:12.756881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c211533f924c58af1b43aba5c9594a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:12.768249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c211533f924c58af1b43aba5c9594a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:12.779915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c211533f924c58af1b43aba5c9594a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:12.780880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c211533f924c58af1b43aba5c9594a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:12.793627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1148/2297] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:12 2022] DELETE /identity/v3/users/82c211533f924c58af1b43aba5c9594a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:12.798590 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1a6d6-9e83-4ac1-984e-cff310d1c1b3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:12.799589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1a6d6-9e83-4ac1-984e-cff310d1c1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:12.801393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1a6d6-9e83-4ac1-984e-cff310d1c1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:12.838476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-24e1a6d6-9e83-4ac1-984e-cff310d1c1b3 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:12.840421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-24e1a6d6-9e83-4ac1-984e-cff310d1c1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:12.930177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:12.969485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1150/2298] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:12.984599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:12.988388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:13.092442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3446b6d-4072-487f-a525-256c390269ca admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:13.106242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874442d1-610e-403d-b55e-d05b8313835a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:13.106878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874442d1-610e-403d-b55e-d05b8313835a service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:13.107156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874442d1-610e-403d-b55e-d05b8313835a service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:13.107531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874442d1-610e-403d-b55e-d05b8313835a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:13.122852 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874442d1-610e-403d-b55e-d05b8313835a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:13.125218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874442d1-610e-403d-b55e-d05b8313835a service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:13.150962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1149/2299] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:38:13.722521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1a6d6-9e83-4ac1-984e-cff310d1c1b3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:13.724316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-24e1a6d6-9e83-4ac1-984e-cff310d1c1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:13.738344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1a6d6-9e83-4ac1-984e-cff310d1c1b3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:13.744871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:13.745605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:13.745938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:13.746221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] PATH_INFO: `/v3/users/85101a4497e94478a64800b06740affd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:13.748254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85101a4497e94478a64800b06740affd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:13.750508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:13.809979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85101a4497e94478a64800b06740affd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:13.821525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85101a4497e94478a64800b06740affd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:13.822274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85101a4497e94478a64800b06740affd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:13.841248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85101a4497e94478a64800b06740affd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:13.850474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85101a4497e94478a64800b06740affd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:13.861921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1151/2300] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:13 2022] DELETE /identity/v3/users/85101a4497e94478a64800b06740affd => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:13.867673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ce6cc-b10d-4cfa-b557-c271b4a9936b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:13.868543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ce6cc-b10d-4cfa-b557-c271b4a9936b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:13.868971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ce6cc-b10d-4cfa-b557-c271b4a9936b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:13.892925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dc8ce6cc-b10d-4cfa-b557-c271b4a9936b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:13.894697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dc8ce6cc-b10d-4cfa-b557-c271b4a9936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:13.990847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1150/2301] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:14.003049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:14.004998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:14.112579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c072b59-b897-49a0-8316-82dfff86e0d1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:14.119099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0df35-9204-41db-b33a-1b5f34dd5ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:14.119748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0df35-9204-41db-b33a-1b5f34dd5ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:14.120033 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0df35-9204-41db-b33a-1b5f34dd5ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:14.120309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0df35-9204-41db-b33a-1b5f34dd5ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:14.132290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0df35-9204-41db-b33a-1b5f34dd5ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:14.133990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0df35-9204-41db-b33a-1b5f34dd5ac7 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:14.149780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1152/2302] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:14.181063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ce6cc-b10d-4cfa-b557-c271b4a9936b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:14.183143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dc8ce6cc-b10d-4cfa-b557-c271b4a9936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:14.205076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ce6cc-b10d-4cfa-b557-c271b4a9936b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:14.213966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:14.215173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:14.218024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:14.218166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] PATH_INFO: `/v3/projects/450ad9198bed4ecfad1f0bb708cd1726` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:14.219783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=450ad9198bed4ecfad1f0bb708cd1726)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:14.222098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:14.277092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '450ad9198bed4ecfad1f0bb708cd1726', 'request_id': 'req-ff978fa8-0fde-491e-bddc-e13621a9be15'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.279707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:14.286650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '450ad9198bed4ecfad1f0bb708cd1726', 'request_id': 'req-ff978fa8-0fde-491e-bddc-e13621a9be15'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.293616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '450ad9198bed4ecfad1f0bb708cd1726', 'request_id': 'req-ff978fa8-0fde-491e-bddc-e13621a9be15'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.304892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] The token cache is being invalidate because project 450ad9198bed4ecfad1f0bb708cd1726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:14.305187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.306653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.308253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1151/2303] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:14 2022] DELETE /identity/v3/projects/450ad9198bed4ecfad1f0bb708cd1726 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:14.312709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0df35-9204-41db-b33a-1b5f34dd5ac7 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:14.323727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0df35-9204-41db-b33a-1b5f34dd5ac7 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:14.397058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:14.397885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:14.398167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:14.398447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] PATH_INFO: `/v3/users/fb06c1fc686f4b23875cac579b7beaa6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:14.400019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb06c1fc686f4b23875cac579b7beaa6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:14.402228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:14.408635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:14.460698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb06c1fc686f4b23875cac579b7beaa6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.475340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb06c1fc686f4b23875cac579b7beaa6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.476219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb06c1fc686f4b23875cac579b7beaa6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.490601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb06c1fc686f4b23875cac579b7beaa6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.499591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb06c1fc686f4b23875cac579b7beaa6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:14.506711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1153/2304] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:14 2022] DELETE /identity/v3/users/fb06c1fc686f4b23875cac579b7beaa6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:14.940598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:14.940598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:14.942322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:14.944624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:14.971843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff978fa8-0fde-491e-bddc-e13621a9be15 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.058202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a19a-d67b-4d2d-a3e0-1ae5824cfeae admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.065094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.066340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.067286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.067992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] PATH_INFO: `/v3/projects/27b5312feab04631a82c2e8445b48a01` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.069979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b5312feab04631a82c2e8445b48a01)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.072913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.074154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.074957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.075481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.075657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] PATH_INFO: `/v3/projects/afdce98bead34925b8192e2f89b9f699` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.076787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] RBAC: Authorizing `identity:delete_project(project_id=afdce98bead34925b8192e2f89b9f699)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.078803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.165305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b5312feab04631a82c2e8445b48a01', 'request_id': 'req-9ce62394-7b0b-454c-8b9d-08396e41b57d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.170393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afdce98bead34925b8192e2f89b9f699', 'request_id': 'req-4bb3a784-8b67-4fea-ac89-384c8642769e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.174652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b5312feab04631a82c2e8445b48a01', 'request_id': 'req-9ce62394-7b0b-454c-8b9d-08396e41b57d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.179966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afdce98bead34925b8192e2f89b9f699', 'request_id': 'req-4bb3a784-8b67-4fea-ac89-384c8642769e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.183371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b5312feab04631a82c2e8445b48a01', 'request_id': 'req-9ce62394-7b0b-454c-8b9d-08396e41b57d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.187639 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afdce98bead34925b8192e2f89b9f699', 'request_id': 'req-4bb3a784-8b67-4fea-ac89-384c8642769e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.202312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] The token cache is being invalidate because project afdce98bead34925b8192e2f89b9f699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:15.202714 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.203474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4bb3a784-8b67-4fea-ac89-384c8642769e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.205290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1152/2305] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:14 2022] DELETE /identity/v3/projects/afdce98bead34925b8192e2f89b9f699 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:15.212577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c8cf8-573a-4491-b942-832bdc8ae935 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.212961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c8cf8-573a-4491-b942-832bdc8ae935 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.213417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c8cf8-573a-4491-b942-832bdc8ae935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.220963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] The token cache is being invalidate because project 27b5312feab04631a82c2e8445b48a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:15.221310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.222163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ce62394-7b0b-454c-8b9d-08396e41b57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.225468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1154/2306] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:14 2022] DELETE /identity/v3/projects/27b5312feab04631a82c2e8445b48a01 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:38:15.232143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e959e-58a3-447f-a9c9-b6854d0ee029 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.232433 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e959e-58a3-447f-a9c9-b6854d0ee029 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.232713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e959e-58a3-447f-a9c9-b6854d0ee029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.247317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6b0c8cf8-573a-4491-b942-832bdc8ae935 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:15.248545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6b0c8cf8-573a-4491-b942-832bdc8ae935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:15.278677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-533e959e-58a3-447f-a9c9-b6854d0ee029 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:15.293600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-533e959e-58a3-447f-a9c9-b6854d0ee029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:15.295514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:15.378726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1155/2307] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:15.380211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1153/2308] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:15.396366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417b98-11d0-4f64-9b83-809228ba7598 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.396821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417b98-11d0-4f64-9b83-809228ba7598 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.397138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417b98-11d0-4f64-9b83-809228ba7598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.399903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d0bd1-72a7-436b-a667-957b35d4e53f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.400302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d0bd1-72a7-436b-a667-957b35d4e53f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.400678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d0bd1-72a7-436b-a667-957b35d4e53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.425803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-519d0bd1-72a7-436b-a667-957b35d4e53f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:15.427143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-519d0bd1-72a7-436b-a667-957b35d4e53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:15.427593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5d417b98-11d0-4f64-9b83-809228ba7598 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:15.428807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5d417b98-11d0-4f64-9b83-809228ba7598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:15.437516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1154/2309] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:15.438284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1156/2310] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:15.443438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d417b98-11d0-4f64-9b83-809228ba7598 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.444804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d0bd1-72a7-436b-a667-957b35d4e53f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.456329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:15.462300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d417b98-11d0-4f64-9b83-809228ba7598 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.462448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d0bd1-72a7-436b-a667-957b35d4e53f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.467920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad036a6-fc25-4740-839f-767734b62ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.468542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad036a6-fc25-4740-839f-767734b62ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.468906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad036a6-fc25-4740-839f-767734b62ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.469140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad036a6-fc25-4740-839f-767734b62ae4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.470125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad036a6-fc25-4740-839f-767734b62ae4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:15.470391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad036a6-fc25-4740-839f-767734b62ae4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.472058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad036a6-fc25-4740-839f-767734b62ae4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.477289 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3650a-0b5d-47aa-acda-d52d2e660508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.478491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3650a-0b5d-47aa-acda-d52d2e660508 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.478981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3650a-0b5d-47aa-acda-d52d2e660508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.479473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3650a-0b5d-47aa-acda-d52d2e660508 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.480455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3650a-0b5d-47aa-acda-d52d2e660508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:15.480980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3650a-0b5d-47aa-acda-d52d2e660508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.482971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3650a-0b5d-47aa-acda-d52d2e660508 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.485823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1157/2311] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:38:15.490257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad036a6-fc25-4740-839f-767734b62ae4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.501513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1155/2312] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:38:15.501723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad036a6-fc25-4740-839f-767734b62ae4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.507325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b8a49-fe61-4b2e-815b-f5a9c8cf7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.507642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3650a-0b5d-47aa-acda-d52d2e660508 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.507964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b8a49-fe61-4b2e-815b-f5a9c8cf7fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.508245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b8a49-fe61-4b2e-815b-f5a9c8cf7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.509229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b8a49-fe61-4b2e-815b-f5a9c8cf7fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.510110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b8a49-fe61-4b2e-815b-f5a9c8cf7fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.512062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b8a49-fe61-4b2e-815b-f5a9c8cf7fa8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.518935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3650a-0b5d-47aa-acda-d52d2e660508 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.524768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280297e6-f51d-43ea-ba6b-88d058295b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.525887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280297e6-f51d-43ea-ba6b-88d058295b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.526346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280297e6-f51d-43ea-ba6b-88d058295b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.526630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280297e6-f51d-43ea-ba6b-88d058295b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.527266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280297e6-f51d-43ea-ba6b-88d058295b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.529169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280297e6-f51d-43ea-ba6b-88d058295b9a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.546516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1158/2313] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:15 2022] POST /identity/v3/projects => generated 374 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:38:15.550485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b8a49-fe61-4b2e-815b-f5a9c8cf7fa8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.559739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1156/2314] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:15 2022] POST /identity/v3/projects => generated 374 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:38:15.563931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280297e6-f51d-43ea-ba6b-88d058295b9a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.628693 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b8a49-fe61-4b2e-815b-f5a9c8cf7fa8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.634190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e633b83-a3a8-4b5c-a0b4-8a53cee46b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.634806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e633b83-a3a8-4b5c-a0b4-8a53cee46b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.635115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e633b83-a3a8-4b5c-a0b4-8a53cee46b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.635311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280297e6-f51d-43ea-ba6b-88d058295b9a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.635516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e633b83-a3a8-4b5c-a0b4-8a53cee46b60 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.636003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e633b83-a3a8-4b5c-a0b4-8a53cee46b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.637964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e633b83-a3a8-4b5c-a0b4-8a53cee46b60 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.640333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2308538-e3f5-437f-81b0-11a5397c8307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.641323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2308538-e3f5-437f-81b0-11a5397c8307 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.641461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2308538-e3f5-437f-81b0-11a5397c8307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.641787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2308538-e3f5-437f-81b0-11a5397c8307 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.642600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2308538-e3f5-437f-81b0-11a5397c8307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.644841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2308538-e3f5-437f-81b0-11a5397c8307 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.674955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1157/2316] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:15 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:38:15.675111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1159/2316] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:15 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:15.675226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0e69-0a88-4d80-8db7-e9ecab23f57a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.675226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0e69-0a88-4d80-8db7-e9ecab23f57a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.675226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0e69-0a88-4d80-8db7-e9ecab23f57a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.675443 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1158/2317] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:38:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:38:15.676334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0e69-0a88-4d80-8db7-e9ecab23f57a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.676587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e633b83-a3a8-4b5c-a0b4-8a53cee46b60 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.687774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e633b83-a3a8-4b5c-a0b4-8a53cee46b60 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.689839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0e69-0a88-4d80-8db7-e9ecab23f57a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.692509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a633a-9513-41b6-a9b8-c2a0033f6e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.693112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a633a-9513-41b6-a9b8-c2a0033f6e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.693393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a633a-9513-41b6-a9b8-c2a0033f6e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.693696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a633a-9513-41b6-a9b8-c2a0033f6e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.694368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a633a-9513-41b6-a9b8-c2a0033f6e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.695143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb32d9-3d3b-49de-b68f-cd33b8e8484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.695861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb32d9-3d3b-49de-b68f-cd33b8e8484a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.696084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a633a-9513-41b6-a9b8-c2a0033f6e56 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.696228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb32d9-3d3b-49de-b68f-cd33b8e8484a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.696589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb32d9-3d3b-49de-b68f-cd33b8e8484a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.697301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb32d9-3d3b-49de-b68f-cd33b8e8484a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.699272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb32d9-3d3b-49de-b68f-cd33b8e8484a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.713473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1160/2318] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:15 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:15.713806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1159/2319] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:15 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:15.713901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd46721-5fa2-43b3-9d67-be7a72e19274 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.713901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd46721-5fa2-43b3-9d67-be7a72e19274 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.713901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd46721-5fa2-43b3-9d67-be7a72e19274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.714354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb32d9-3d3b-49de-b68f-cd33b8e8484a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.726523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb32d9-3d3b-49de-b68f-cd33b8e8484a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.735011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e71ca-877e-410b-afc7-e5c330361ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.735964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e71ca-877e-410b-afc7-e5c330361ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.736213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fbd46721-5fa2-43b3-9d67-be7a72e19274 None None] MFA Rules not processed for user `4b181d38588d44d4bb72fa319d2ba2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:15.736323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e71ca-877e-410b-afc7-e5c330361ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.736513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e71ca-877e-410b-afc7-e5c330361ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.737193 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e71ca-877e-410b-afc7-e5c330361ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.737428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fbd46721-5fa2-43b3-9d67-be7a72e19274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:15.740177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e71ca-877e-410b-afc7-e5c330361ff7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.761157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1160/2320] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:15 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:15.761157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e71ca-877e-410b-afc7-e5c330361ff7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.771496 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e71ca-877e-410b-afc7-e5c330361ff7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.776197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734feabb-c6ca-4d5f-accd-cf5d07ea5c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.776796 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734feabb-c6ca-4d5f-accd-cf5d07ea5c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.777068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734feabb-c6ca-4d5f-accd-cf5d07ea5c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.777337 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734feabb-c6ca-4d5f-accd-cf5d07ea5c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.778062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734feabb-c6ca-4d5f-accd-cf5d07ea5c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.779650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734feabb-c6ca-4d5f-accd-cf5d07ea5c49 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.790954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1161/2321] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:15 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:15.791621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734feabb-c6ca-4d5f-accd-cf5d07ea5c49 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.795431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-734feabb-c6ca-4d5f-accd-cf5d07ea5c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:15.799096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:15.810633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734feabb-c6ca-4d5f-accd-cf5d07ea5c49 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.815294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.815952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.816115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.816403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] PATH_INFO: `/v3/users/bcea72e4fc77431a96e7acf9d68eb083` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.817485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcea72e4fc77431a96e7acf9d68eb083)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.818071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1161/2322] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:38:15 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:15.819716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.820246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd46721-5fa2-43b3-9d67-be7a72e19274 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.832177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd46721-5fa2-43b3-9d67-be7a72e19274 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.836844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151334e-0691-473d-946c-641490dd30b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:15.837508 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151334e-0691-473d-946c-641490dd30b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:15.837821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151334e-0691-473d-946c-641490dd30b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:15.838096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151334e-0691-473d-946c-641490dd30b9 admin admin] PATH_INFO: `/v3/projects/0209b8de4b2b43f886ef0c298c46cd3a/users/387c3f2fc4e04c1ca7231a35ab4b0eff/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:15.855112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151334e-0691-473d-946c-641490dd30b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0209b8de4b2b43f886ef0c298c46cd3a, user_id=387c3f2fc4e04c1ca7231a35ab4b0eff, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:15.858003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151334e-0691-473d-946c-641490dd30b9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:15.872300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1162/2323] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:15 2022] PUT /identity/v3/projects/0209b8de4b2b43f886ef0c298c46cd3a/users/387c3f2fc4e04c1ca7231a35ab4b0eff/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:15.872468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcea72e4fc77431a96e7acf9d68eb083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.872872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151334e-0691-473d-946c-641490dd30b9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.887048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcea72e4fc77431a96e7acf9d68eb083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.893932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcea72e4fc77431a96e7acf9d68eb083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.902413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcea72e4fc77431a96e7acf9d68eb083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.903498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcea72e4fc77431a96e7acf9d68eb083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:15.905027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:15.911131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1162/2324] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:15 2022] DELETE /identity/v3/users/bcea72e4fc77431a96e7acf9d68eb083 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:15.912434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:15.914275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:15.945270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151334e-0691-473d-946c-641490dd30b9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:15.999818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723038a4-a051-4e4b-829b-9af39647c70a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:16.005395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9ef0-28df-4bf5-9d74-85233b7e2ba9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:16.006145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9ef0-28df-4bf5-9d74-85233b7e2ba9 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.006494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9ef0-28df-4bf5-9d74-85233b7e2ba9 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.006830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9ef0-28df-4bf5-9d74-85233b7e2ba9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.006952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba88730-56e9-401a-a01f-ada09386c523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:16.008961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba88730-56e9-401a-a01f-ada09386c523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.009255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba88730-56e9-401a-a01f-ada09386c523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.009532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba88730-56e9-401a-a01f-ada09386c523 admin admin] PATH_INFO: `/v3/projects/179e3dff6d83482f94652fdf036ef08a/users/0f7e9fe846df411f90eea3c298093ff2/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.009741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-075d9ef0-28df-4bf5-9d74-85233b7e2ba9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:16.022528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d9ef0-28df-4bf5-9d74-85233b7e2ba9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:16.025317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d9ef0-28df-4bf5-9d74-85233b7e2ba9 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:16.029444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba88730-56e9-401a-a01f-ada09386c523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=179e3dff6d83482f94652fdf036ef08a, user_id=0f7e9fe846df411f90eea3c298093ff2, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:16.033128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba88730-56e9-401a-a01f-ada09386c523 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:16.041117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1163/2325] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:15 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:16.042473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad7659-d620-45b6-a20c-b8a395101956 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.042834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad7659-d620-45b6-a20c-b8a395101956 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.043187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad7659-d620-45b6-a20c-b8a395101956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.051309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1163/2326] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:15 2022] PUT /identity/v3/projects/179e3dff6d83482f94652fdf036ef08a/users/0f7e9fe846df411f90eea3c298093ff2/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:16.063807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807dc59-41ec-4c2b-8aca-72099fdded5b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.064102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807dc59-41ec-4c2b-8aca-72099fdded5b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.065013 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807dc59-41ec-4c2b-8aca-72099fdded5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.079732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ebad7659-d620-45b6-a20c-b8a395101956 None None] MFA Rules not processed for user `387c3f2fc4e04c1ca7231a35ab4b0eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:16.081844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ebad7659-d620-45b6-a20c-b8a395101956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:16.097990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3807dc59-41ec-4c2b-8aca-72099fdded5b None None] MFA Rules not processed for user `0f7e9fe846df411f90eea3c298093ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:16.099501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3807dc59-41ec-4c2b-8aca-72099fdded5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:16.154400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:16.189736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1164/2327] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:16.190898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807dc59-41ec-4c2b-8aca-72099fdded5b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:16.192745 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3807dc59-41ec-4c2b-8aca-72099fdded5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:16.201114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1164/2328] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:16.205649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad7659-d620-45b6-a20c-b8a395101956 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:16.207324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ebad7659-d620-45b6-a20c-b8a395101956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:16.288361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807dc59-41ec-4c2b-8aca-72099fdded5b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:16.293134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:16.293809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.294089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.294364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] PATH_INFO: `/v3/projects/df3925a7fb6c4d099c4f160d42870a3b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.295489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3925a7fb6c4d099c4f160d42870a3b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:16.297502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:16.301018 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad7659-d620-45b6-a20c-b8a395101956 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:16.304913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5807d0-78a1-474d-9398-71bca0a1d12d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:16.306895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5807d0-78a1-474d-9398-71bca0a1d12d service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.307175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5807d0-78a1-474d-9398-71bca0a1d12d service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.307444 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5807d0-78a1-474d-9398-71bca0a1d12d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.317893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5807d0-78a1-474d-9398-71bca0a1d12d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:16.319292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5807d0-78a1-474d-9398-71bca0a1d12d service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:16.331568 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1165/2329] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:16.355048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3925a7fb6c4d099c4f160d42870a3b', 'request_id': 'req-8af8036d-4935-4422-b8cc-d48abf0bea67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:16.362544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3925a7fb6c4d099c4f160d42870a3b', 'request_id': 'req-8af8036d-4935-4422-b8cc-d48abf0bea67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:16.369343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3925a7fb6c4d099c4f160d42870a3b', 'request_id': 'req-8af8036d-4935-4422-b8cc-d48abf0bea67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:16.387056 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] The token cache is being invalidate because project df3925a7fb6c4d099c4f160d42870a3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:16.387386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:16.388079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:16.394676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1165/2330] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:16 2022] DELETE /identity/v3/projects/df3925a7fb6c4d099c4f160d42870a3b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:16.396555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe99630-943d-4d0f-bb9e-dca2454a043f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.396900 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe99630-943d-4d0f-bb9e-dca2454a043f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.397220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe99630-943d-4d0f-bb9e-dca2454a043f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.422439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ffe99630-943d-4d0f-bb9e-dca2454a043f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:16.423667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ffe99630-943d-4d0f-bb9e-dca2454a043f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:16.442967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:16.490439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:16.492250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:16.527968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1166/2331] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:16.529913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a02f-1ce7-428c-94fc-83b581ea3cf5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.530155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a02f-1ce7-428c-94fc-83b581ea3cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.530436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a02f-1ce7-428c-94fc-83b581ea3cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.558700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-66e9a02f-1ce7-428c-94fc-83b581ea3cf5 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:16.560055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-66e9a02f-1ce7-428c-94fc-83b581ea3cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:16.573932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1167/2332] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:16.578241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a02f-1ce7-428c-94fc-83b581ea3cf5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:16.590255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8036d-4935-4422-b8cc-d48abf0bea67 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:16.591616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a02f-1ce7-428c-94fc-83b581ea3cf5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:16.594751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce4abb-6932-433b-8755-b0c942a13dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:16.595530 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce4abb-6932-433b-8755-b0c942a13dee service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.595957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce4abb-6932-433b-8755-b0c942a13dee service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.596282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce4abb-6932-433b-8755-b0c942a13dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.596651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ce525-8771-4190-8b8c-f983b9b1bd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:16.597336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ce525-8771-4190-8b8c-f983b9b1bd40 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.597858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ce525-8771-4190-8b8c-f983b9b1bd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.598207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ce525-8771-4190-8b8c-f983b9b1bd40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.598576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-36ce4abb-6932-433b-8755-b0c942a13dee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:16.599175 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ce525-8771-4190-8b8c-f983b9b1bd40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:16.599500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ce525-8771-4190-8b8c-f983b9b1bd40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:16.601669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ce525-8771-4190-8b8c-f983b9b1bd40 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:16.614790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1168/2333] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:38:16.619995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ce525-8771-4190-8b8c-f983b9b1bd40 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:16.630457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ce525-8771-4190-8b8c-f983b9b1bd40 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:16.637197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f44d7-0ed3-437b-8ef4-3fe511ddca76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:16.643013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f44d7-0ed3-437b-8ef4-3fe511ddca76 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.643517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f44d7-0ed3-437b-8ef4-3fe511ddca76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.643860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f44d7-0ed3-437b-8ef4-3fe511ddca76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.644626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f44d7-0ed3-437b-8ef4-3fe511ddca76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:16.646672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f44d7-0ed3-437b-8ef4-3fe511ddca76 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:16.652462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:16.690342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1169/2334] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:16 2022] POST /identity/v3/projects => generated 384 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:16.691401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f44d7-0ed3-437b-8ef4-3fe511ddca76 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:16.734469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce4abb-6932-433b-8755-b0c942a13dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:16.738021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce4abb-6932-433b-8755-b0c942a13dee service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:16.751807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1166/2335] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:38:16.833318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f44d7-0ed3-437b-8ef4-3fe511ddca76 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:16.841445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039932f2-672e-4081-914f-b1517ce237c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:16.842562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039932f2-672e-4081-914f-b1517ce237c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.850951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039932f2-672e-4081-914f-b1517ce237c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.851381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039932f2-672e-4081-914f-b1517ce237c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.853093 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039932f2-672e-4081-914f-b1517ce237c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:16.856694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039932f2-672e-4081-914f-b1517ce237c1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:16.895782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1170/2336] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:16 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:16.897232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce4abb-6932-433b-8755-b0c942a13dee service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:16.917704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce4abb-6932-433b-8755-b0c942a13dee service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:16.929162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b62986-d52d-40de-8ea3-75e1147cb2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:16.932474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b62986-d52d-40de-8ea3-75e1147cb2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:16.933119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b62986-d52d-40de-8ea3-75e1147cb2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:16.933601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b62986-d52d-40de-8ea3-75e1147cb2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:16.938476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b62986-d52d-40de-8ea3-75e1147cb2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:16.943012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b62986-d52d-40de-8ea3-75e1147cb2c7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:16.968250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1167/2337] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:16 2022] GET /identity/v3/roles => generated 1427 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:16.972177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039932f2-672e-4081-914f-b1517ce237c1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:16.994488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039932f2-672e-4081-914f-b1517ce237c1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:17.002655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d8ea1-9101-4224-979d-b664cc854cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:17.003648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8ea1-9101-4224-979d-b664cc854cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:17.003951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8ea1-9101-4224-979d-b664cc854cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:17.004481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8ea1-9101-4224-979d-b664cc854cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:17.005282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d8ea1-9101-4224-979d-b664cc854cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:17.007224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d8ea1-9101-4224-979d-b664cc854cfa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:17.028750 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1171/2338] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:16 2022] GET /identity/v3/roles => generated 1427 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:38:17.035661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b62986-d52d-40de-8ea3-75e1147cb2c7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:17.057702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b62986-d52d-40de-8ea3-75e1147cb2c7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:17.065714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd8170-23c3-4264-ba51-2ad10975a39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:17.066400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd8170-23c3-4264-ba51-2ad10975a39e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:17.066680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd8170-23c3-4264-ba51-2ad10975a39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:17.066956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd8170-23c3-4264-ba51-2ad10975a39e admin admin] PATH_INFO: `/v3/projects/1cf7216d5c904b24a9665755720cf287/users/d3848643d2c44951b78814ccf7d504af/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:17.118389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd8170-23c3-4264-ba51-2ad10975a39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf7216d5c904b24a9665755720cf287, user_id=d3848643d2c44951b78814ccf7d504af, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:17.131628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd8170-23c3-4264-ba51-2ad10975a39e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:17.163953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1168/2339] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:17 2022] PUT /identity/v3/projects/1cf7216d5c904b24a9665755720cf287/users/d3848643d2c44951b78814ccf7d504af/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:17.165593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb293-0ea3-4f05-90c7-7e37f48c4d47 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:17.166538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb293-0ea3-4f05-90c7-7e37f48c4d47 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:17.167516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb293-0ea3-4f05-90c7-7e37f48c4d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:17.214591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9dcdb293-0ea3-4f05-90c7-7e37f48c4d47 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:17.216227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9dcdb293-0ea3-4f05-90c7-7e37f48c4d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:17.331971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:17.372972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1172/2340] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:17.388404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd8170-23c3-4264-ba51-2ad10975a39e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:17.392888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-28fd8170-23c3-4264-ba51-2ad10975a39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:17.603653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd8170-23c3-4264-ba51-2ad10975a39e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:17.614566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869d59a-4714-4368-b00e-3f3815ccef62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:17.615396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869d59a-4714-4368-b00e-3f3815ccef62 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:17.621343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869d59a-4714-4368-b00e-3f3815ccef62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:17.621343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869d59a-4714-4368-b00e-3f3815ccef62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:17.668630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869d59a-4714-4368-b00e-3f3815ccef62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:17.670468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869d59a-4714-4368-b00e-3f3815ccef62 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:17.690700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1169/2341] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:19.963113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb293-0ea3-4f05-90c7-7e37f48c4d47 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:19.964604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9dcdb293-0ea3-4f05-90c7-7e37f48c4d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:19.979283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb293-0ea3-4f05-90c7-7e37f48c4d47 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:19.988275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:19.989216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:19.989698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:19.990122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] PATH_INFO: `/v3/users/0f7e9fe846df411f90eea3c298093ff2` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:19.991513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7e9fe846df411f90eea3c298093ff2)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:19.994246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:20.075194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7e9fe846df411f90eea3c298093ff2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.096205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7e9fe846df411f90eea3c298093ff2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.106732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7e9fe846df411f90eea3c298093ff2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.132037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7e9fe846df411f90eea3c298093ff2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.132811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7e9fe846df411f90eea3c298093ff2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.149491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1173/2342] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:19 2022] DELETE /identity/v3/users/0f7e9fe846df411f90eea3c298093ff2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:20.152553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04737f-eec0-4faa-9ada-e415a237e786 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:20.153052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04737f-eec0-4faa-9ada-e415a237e786 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:20.153238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04737f-eec0-4faa-9ada-e415a237e786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:20.226188 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5d04737f-eec0-4faa-9ada-e415a237e786 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:20.228076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5d04737f-eec0-4faa-9ada-e415a237e786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:20.314199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1170/2343] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:20.346969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:20.348542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:20.375812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04737f-eec0-4faa-9ada-e415a237e786 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:20.378011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5d04737f-eec0-4faa-9ada-e415a237e786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:20.399918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04737f-eec0-4faa-9ada-e415a237e786 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:20.407695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:20.409140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:20.409810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:20.410576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] PATH_INFO: `/v3/users/9282e3834a754f69960a60ff290d9bac` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:20.412453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9282e3834a754f69960a60ff290d9bac)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:20.419233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:20.472564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e24e3-0b3e-4f42-bf57-2fa3b878e213 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:20.481879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:20.485356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8d525-d2a2-4184-8e8e-01408bcae6f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:20.487557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8d525-d2a2-4184-8e8e-01408bcae6f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:20.488051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8d525-d2a2-4184-8e8e-01408bcae6f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:20.488765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8d525-d2a2-4184-8e8e-01408bcae6f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:20.502478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9282e3834a754f69960a60ff290d9bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.507821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8d525-d2a2-4184-8e8e-01408bcae6f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:20.511505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8d525-d2a2-4184-8e8e-01408bcae6f6 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:20.516289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9282e3834a754f69960a60ff290d9bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.518830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9282e3834a754f69960a60ff290d9bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.528238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1174/2344] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:20.540843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9282e3834a754f69960a60ff290d9bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.554011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9282e3834a754f69960a60ff290d9bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:20.574912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1171/2345] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:20 2022] DELETE /identity/v3/users/9282e3834a754f69960a60ff290d9bac => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:38:20.579689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8f5e9-233a-4fc0-8028-df9db29e9021 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:20.580094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8f5e9-233a-4fc0-8028-df9db29e9021 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:20.580441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8f5e9-233a-4fc0-8028-df9db29e9021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:20.615292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5af8f5e9-233a-4fc0-8028-df9db29e9021 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:20.616799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5af8f5e9-233a-4fc0-8028-df9db29e9021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:20.792204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1175/2346] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:20.874336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:20.875867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:20.966132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8f5e9-233a-4fc0-8028-df9db29e9021 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:20.970466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5af8f5e9-233a-4fc0-8028-df9db29e9021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:21.028451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8f5e9-233a-4fc0-8028-df9db29e9021 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:21.043629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:21.044408 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.044690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.044968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] PATH_INFO: `/v3/projects/179e3dff6d83482f94652fdf036ef08a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.046445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] RBAC: Authorizing `identity:delete_project(project_id=179e3dff6d83482f94652fdf036ef08a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:21.050151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:21.052378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f1baa-418e-465b-a7fc-cf550ee8302d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:21.056559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef5f55-5432-413e-bce2-656d05e1f88f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:21.057947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef5f55-5432-413e-bce2-656d05e1f88f service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.058186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef5f55-5432-413e-bce2-656d05e1f88f service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.058458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef5f55-5432-413e-bce2-656d05e1f88f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.068883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:21.090875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef5f55-5432-413e-bce2-656d05e1f88f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:21.093408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef5f55-5432-413e-bce2-656d05e1f88f service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:21.117165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1172/2347] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:21.142140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179e3dff6d83482f94652fdf036ef08a', 'request_id': 'req-a43640e1-da10-4e04-a2bb-d317191c164d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:21.150249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179e3dff6d83482f94652fdf036ef08a', 'request_id': 'req-a43640e1-da10-4e04-a2bb-d317191c164d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:21.158318 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179e3dff6d83482f94652fdf036ef08a', 'request_id': 'req-a43640e1-da10-4e04-a2bb-d317191c164d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:21.175630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] The token cache is being invalidate because project 179e3dff6d83482f94652fdf036ef08a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:21.176507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:21.177119 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a43640e1-da10-4e04-a2bb-d317191c164d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:21.179118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1176/2348] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:20 2022] DELETE /identity/v3/projects/179e3dff6d83482f94652fdf036ef08a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:21.184709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a3806-5a3f-4632-ac31-b0c7a199949a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.185221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a3806-5a3f-4632-ac31-b0c7a199949a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.185792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a3806-5a3f-4632-ac31-b0c7a199949a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.222982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cb9a3806-5a3f-4632-ac31-b0c7a199949a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:21.224306 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb9a3806-5a3f-4632-ac31-b0c7a199949a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:21.322540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1173/2349] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:21.331520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4054b-3431-453d-9368-0dfcc19fce02 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.331810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4054b-3431-453d-9368-0dfcc19fce02 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.332078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4054b-3431-453d-9368-0dfcc19fce02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.357229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-07b4054b-3431-453d-9368-0dfcc19fce02 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:21.375007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-07b4054b-3431-453d-9368-0dfcc19fce02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:21.388023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1177/2350] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:21.392572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a3806-5a3f-4632-ac31-b0c7a199949a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:21.405046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a3806-5a3f-4632-ac31-b0c7a199949a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:21.411811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0e735-f7a8-45f7-af0c-265dad8a334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:21.412886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0e735-f7a8-45f7-af0c-265dad8a334b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.413272 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0e735-f7a8-45f7-af0c-265dad8a334b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.413695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0e735-f7a8-45f7-af0c-265dad8a334b admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.414500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0e735-f7a8-45f7-af0c-265dad8a334b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:21.414915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0e735-f7a8-45f7-af0c-265dad8a334b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:21.417240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0e735-f7a8-45f7-af0c-265dad8a334b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:21.432764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1174/2351] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:38:21.437356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4054b-3431-453d-9368-0dfcc19fce02 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:21.449520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4054b-3431-453d-9368-0dfcc19fce02 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:21.455524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c7d4e-ab48-48e3-bb05-c6b3762fcf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:21.456505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c7d4e-ab48-48e3-bb05-c6b3762fcf24 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.457202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c7d4e-ab48-48e3-bb05-c6b3762fcf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.457596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c7d4e-ab48-48e3-bb05-c6b3762fcf24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.458361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c7d4e-ab48-48e3-bb05-c6b3762fcf24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:21.460400 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c7d4e-ab48-48e3-bb05-c6b3762fcf24 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:21.501052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1178/2352] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:21 2022] POST /identity/v3/projects => generated 386 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:38:21.505465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0e735-f7a8-45f7-af0c-265dad8a334b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:21.527266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0e735-f7a8-45f7-af0c-265dad8a334b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:21.594540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:21.653722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62365d-94e0-448c-8876-c7ed0b322799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:21.654356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62365d-94e0-448c-8876-c7ed0b322799 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.654636 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62365d-94e0-448c-8876-c7ed0b322799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.656421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62365d-94e0-448c-8876-c7ed0b322799 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.657409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62365d-94e0-448c-8876-c7ed0b322799 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:21.659400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62365d-94e0-448c-8876-c7ed0b322799 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:21.712669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c7d4e-ab48-48e3-bb05-c6b3762fcf24 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:21.714802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1175/2353] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:21 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:21.736956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c7d4e-ab48-48e3-bb05-c6b3762fcf24 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:21.747869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538e42d-9d05-426a-8eb7-17088dbbf6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:21.748613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538e42d-9d05-426a-8eb7-17088dbbf6ab admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.748961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538e42d-9d05-426a-8eb7-17088dbbf6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.749305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538e42d-9d05-426a-8eb7-17088dbbf6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.750159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538e42d-9d05-426a-8eb7-17088dbbf6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:21.751949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538e42d-9d05-426a-8eb7-17088dbbf6ab admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:21.783509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1179/2354] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:21 2022] GET /identity/v3/roles => generated 1427 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:21.785157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62365d-94e0-448c-8876-c7ed0b322799 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:21.803918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62365d-94e0-448c-8876-c7ed0b322799 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:21.822657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4584e2-4662-44c8-b61a-03bc84c08479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:21.823285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4584e2-4662-44c8-b61a-03bc84c08479 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.823694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4584e2-4662-44c8-b61a-03bc84c08479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.823971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4584e2-4662-44c8-b61a-03bc84c08479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.824632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4584e2-4662-44c8-b61a-03bc84c08479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:21.826318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4584e2-4662-44c8-b61a-03bc84c08479 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:21.849941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1176/2355] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:21 2022] GET /identity/v3/roles => generated 1427 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:21.851183 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538e42d-9d05-426a-8eb7-17088dbbf6ab admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:21.865389 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538e42d-9d05-426a-8eb7-17088dbbf6ab admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:21.878577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73c254-1b05-4e63-b9db-d9cd913f9d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:21.879266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73c254-1b05-4e63-b9db-d9cd913f9d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.879545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73c254-1b05-4e63-b9db-d9cd913f9d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.879823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73c254-1b05-4e63-b9db-d9cd913f9d35 admin admin] PATH_INFO: `/v3/projects/33e87897fc5c47bcba76036c7a60b327/users/4760be60dd434c6bad44e911d5621f67/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:21.916192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73c254-1b05-4e63-b9db-d9cd913f9d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e87897fc5c47bcba76036c7a60b327, user_id=4760be60dd434c6bad44e911d5621f67, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:21.918674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73c254-1b05-4e63-b9db-d9cd913f9d35 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:21.953614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1180/2356] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:21 2022] PUT /identity/v3/projects/33e87897fc5c47bcba76036c7a60b327/users/4760be60dd434c6bad44e911d5621f67/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:21.964153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ef1bb-c020-4d96-8177-b32eb4939d8f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:21.964461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ef1bb-c020-4d96-8177-b32eb4939d8f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:21.964723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ef1bb-c020-4d96-8177-b32eb4939d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:22.000242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e68ef1bb-c020-4d96-8177-b32eb4939d8f None None] MFA Rules not processed for user `4760be60dd434c6bad44e911d5621f67`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:22.001957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e68ef1bb-c020-4d96-8177-b32eb4939d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:22.016739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73c254-1b05-4e63-b9db-d9cd913f9d35 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:22.018865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dd73c254-1b05-4e63-b9db-d9cd913f9d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:22.185037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73c254-1b05-4e63-b9db-d9cd913f9d35 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:22.189769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1177/2357] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:22.195176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:22.197097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:22.197517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:22.197839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] PATH_INFO: `/v3/projects/4a05b1fd52434d32bbf2333c73775509` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:22.199299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a05b1fd52434d32bbf2333c73775509)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:22.202352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:22.204320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ef1bb-c020-4d96-8177-b32eb4939d8f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:22.205882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e68ef1bb-c020-4d96-8177-b32eb4939d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:22.290370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:22.298522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ef1bb-c020-4d96-8177-b32eb4939d8f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:22.335332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a05b1fd52434d32bbf2333c73775509', 'request_id': 'req-04af818e-03e2-4ee7-86af-77a7167a3030'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:22.350437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a05b1fd52434d32bbf2333c73775509', 'request_id': 'req-04af818e-03e2-4ee7-86af-77a7167a3030'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:22.380238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a05b1fd52434d32bbf2333c73775509', 'request_id': 'req-04af818e-03e2-4ee7-86af-77a7167a3030'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:22.384428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc2891-c359-4cbe-8e21-6ac57fb9c28f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:22.385359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc2891-c359-4cbe-8e21-6ac57fb9c28f service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:22.385771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc2891-c359-4cbe-8e21-6ac57fb9c28f service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:22.386165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc2891-c359-4cbe-8e21-6ac57fb9c28f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:22.395679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] The token cache is being invalidate because project 4a05b1fd52434d32bbf2333c73775509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:22.396224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:22.397046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04af818e-03e2-4ee7-86af-77a7167a3030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:22.400180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1181/2358] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:22 2022] DELETE /identity/v3/projects/4a05b1fd52434d32bbf2333c73775509 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:22.401950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9aa2a-7f8a-45f1-b4f9-51242e3bd773 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:22.402459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9aa2a-7f8a-45f1-b4f9-51242e3bd773 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:22.403099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9aa2a-7f8a-45f1-b4f9-51242e3bd773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:22.409422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:22.454912 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9ab9aa2a-7f8a-45f1-b4f9-51242e3bd773 None None] MFA Rules not processed for user `75de510214004bbeab187d65bf178735`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:22.458208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9ab9aa2a-7f8a-45f1-b4f9-51242e3bd773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:22.472547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc2891-c359-4cbe-8e21-6ac57fb9c28f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:22.474219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc2891-c359-4cbe-8e21-6ac57fb9c28f service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:22.513964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1178/2359] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:22 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:38:22.515464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0fbb5-ea1f-4630-9b5a-fc231ebd33f2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:22.515755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0fbb5-ea1f-4630-9b5a-fc231ebd33f2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:22.516034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0fbb5-ea1f-4630-9b5a-fc231ebd33f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:22.546536 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5da0fbb5-ea1f-4630-9b5a-fc231ebd33f2 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:22.548474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5da0fbb5-ea1f-4630-9b5a-fc231ebd33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:22.585628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1182/2360] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:22.640302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1179/2361] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:22.665192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76217a77-6cac-4c26-8cfb-39bcfa2c0c7b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:22.665677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76217a77-6cac-4c26-8cfb-39bcfa2c0c7b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:22.666120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76217a77-6cac-4c26-8cfb-39bcfa2c0c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:22.701321 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-76217a77-6cac-4c26-8cfb-39bcfa2c0c7b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:22.702635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-76217a77-6cac-4c26-8cfb-39bcfa2c0c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:22.720514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1183/2362] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:22.720747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0fbb5-ea1f-4630-9b5a-fc231ebd33f2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:22.733686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0fbb5-ea1f-4630-9b5a-fc231ebd33f2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:22.740814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff1187-0c26-4d52-bb91-838acf736610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:22.741943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1187-0c26-4d52-bb91-838acf736610 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:22.742318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1187-0c26-4d52-bb91-838acf736610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:22.742721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1187-0c26-4d52-bb91-838acf736610 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:22.744149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff1187-0c26-4d52-bb91-838acf736610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:22.744582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff1187-0c26-4d52-bb91-838acf736610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:22.746957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff1187-0c26-4d52-bb91-838acf736610 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:22.765029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1180/2363] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:38:22.769882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76217a77-6cac-4c26-8cfb-39bcfa2c0c7b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:22.785274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76217a77-6cac-4c26-8cfb-39bcfa2c0c7b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:22.794924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481baa2-f428-47d0-9d7b-dc769e61bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:22.795600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481baa2-f428-47d0-9d7b-dc769e61bb72 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:22.795882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481baa2-f428-47d0-9d7b-dc769e61bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:22.796161 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481baa2-f428-47d0-9d7b-dc769e61bb72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:22.796842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481baa2-f428-47d0-9d7b-dc769e61bb72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:22.799996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481baa2-f428-47d0-9d7b-dc769e61bb72 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:22.858566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1184/2364] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:22 2022] POST /identity/v3/projects => generated 382 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:22.861494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff1187-0c26-4d52-bb91-838acf736610 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:22.873958 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff1187-0c26-4d52-bb91-838acf736610 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:23.025858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f5cb6-8b1b-4e8b-b09f-f5cb8c2d56c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:23.026653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f5cb6-8b1b-4e8b-b09f-f5cb8c2d56c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:23.027040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f5cb6-8b1b-4e8b-b09f-f5cb8c2d56c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:23.027483 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f5cb6-8b1b-4e8b-b09f-f5cb8c2d56c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:23.028226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f5cb6-8b1b-4e8b-b09f-f5cb8c2d56c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:23.034778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f5cb6-8b1b-4e8b-b09f-f5cb8c2d56c1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:23.075357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1181/2365] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:22 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:23.077128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481baa2-f428-47d0-9d7b-dc769e61bb72 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:23.095280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481baa2-f428-47d0-9d7b-dc769e61bb72 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:23.107470 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55157d-cd72-4362-993b-5d2975dc9316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:23.109782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55157d-cd72-4362-993b-5d2975dc9316 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:23.110700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55157d-cd72-4362-993b-5d2975dc9316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:23.111264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55157d-cd72-4362-993b-5d2975dc9316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:23.112160 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55157d-cd72-4362-993b-5d2975dc9316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:23.114579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55157d-cd72-4362-993b-5d2975dc9316 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:23.134301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1185/2366] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:23 2022] GET /identity/v3/roles => generated 1427 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:23.135349 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f5cb6-8b1b-4e8b-b09f-f5cb8c2d56c1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:23.146773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f5cb6-8b1b-4e8b-b09f-f5cb8c2d56c1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:23.154128 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cd51f-456f-48c1-9df1-465011448ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:23.155049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cd51f-456f-48c1-9df1-465011448ace admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:23.155340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cd51f-456f-48c1-9df1-465011448ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:23.155608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cd51f-456f-48c1-9df1-465011448ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:23.156252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cd51f-456f-48c1-9df1-465011448ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:23.157894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cd51f-456f-48c1-9df1-465011448ace admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:23.173350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1182/2367] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:23 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:23.176887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55157d-cd72-4362-993b-5d2975dc9316 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:23.191255 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55157d-cd72-4362-993b-5d2975dc9316 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:23.198074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc1f53-3110-47b2-b6ab-e40a356839c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:23.199052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc1f53-3110-47b2-b6ab-e40a356839c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:23.199453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc1f53-3110-47b2-b6ab-e40a356839c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:23.200201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc1f53-3110-47b2-b6ab-e40a356839c5 admin admin] PATH_INFO: `/v3/projects/6695ac82241740f5ac5f73b4bcdac5f5/users/66366eb7b7044786b23d08676c14670f/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:23.245872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc1f53-3110-47b2-b6ab-e40a356839c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6695ac82241740f5ac5f73b4bcdac5f5, user_id=66366eb7b7044786b23d08676c14670f, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:23.250395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc1f53-3110-47b2-b6ab-e40a356839c5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:23.278891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1186/2368] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:23 2022] PUT /identity/v3/projects/6695ac82241740f5ac5f73b4bcdac5f5/users/66366eb7b7044786b23d08676c14670f/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:23.284122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c8589-34d2-4d76-a628-4dd7a0778a0f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:23.284627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c8589-34d2-4d76-a628-4dd7a0778a0f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:23.284736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c8589-34d2-4d76-a628-4dd7a0778a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:23.313004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a58c8589-34d2-4d76-a628-4dd7a0778a0f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:23.315741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a58c8589-34d2-4d76-a628-4dd7a0778a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:23.340071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:23.426002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1183/2369] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:23.438358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc1f53-3110-47b2-b6ab-e40a356839c5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:23.440441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-29cc1f53-3110-47b2-b6ab-e40a356839c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:23.532875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc1f53-3110-47b2-b6ab-e40a356839c5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:23.538259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef549a5e-d8b7-4c21-b7bf-47926ce37142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:23.538977 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549a5e-d8b7-4c21-b7bf-47926ce37142 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:23.539698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549a5e-d8b7-4c21-b7bf-47926ce37142 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:23.540680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549a5e-d8b7-4c21-b7bf-47926ce37142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:23.558775 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef549a5e-d8b7-4c21-b7bf-47926ce37142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:23.560233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef549a5e-d8b7-4c21-b7bf-47926ce37142 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:23.574502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1187/2370] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:24.935100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e2e85-0eba-4ce7-bd08-32dabfee6376 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:24.936018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e2e85-0eba-4ce7-bd08-32dabfee6376 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:24.936589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e2e85-0eba-4ce7-bd08-32dabfee6376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:25.016820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8f1e2e85-0eba-4ce7-bd08-32dabfee6376 None None] MFA Rules not processed for user `66366eb7b7044786b23d08676c14670f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:25.021904 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8f1e2e85-0eba-4ce7-bd08-32dabfee6376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:25.177514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1184/2371] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:25.188633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef549a5e-d8b7-4c21-b7bf-47926ce37142 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:25.190194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ef549a5e-d8b7-4c21-b7bf-47926ce37142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:25.206217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:25.356394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef549a5e-d8b7-4c21-b7bf-47926ce37142 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:25.363972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b86d1-cdf5-4080-a7df-e6cf10cfc5af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:25.366157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b86d1-cdf5-4080-a7df-e6cf10cfc5af service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:25.366609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b86d1-cdf5-4080-a7df-e6cf10cfc5af service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:25.366983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b86d1-cdf5-4080-a7df-e6cf10cfc5af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:25.398444 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b86d1-cdf5-4080-a7df-e6cf10cfc5af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:25.400438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b86d1-cdf5-4080-a7df-e6cf10cfc5af service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:25.414846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1188/2372] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:25.755753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e2e85-0eba-4ce7-bd08-32dabfee6376 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:25.757644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8f1e2e85-0eba-4ce7-bd08-32dabfee6376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:25.804011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e2e85-0eba-4ce7-bd08-32dabfee6376 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:25.815973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:25.818421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:25.818965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:25.819478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] PATH_INFO: `/v3/users/4760be60dd434c6bad44e911d5621f67` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:25.821859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=4760be60dd434c6bad44e911d5621f67)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:25.824850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:25.922030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4760be60dd434c6bad44e911d5621f67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:25.942785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4760be60dd434c6bad44e911d5621f67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:25.945008 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4760be60dd434c6bad44e911d5621f67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:25.967536 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4760be60dd434c6bad44e911d5621f67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:25.988659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4760be60dd434c6bad44e911d5621f67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:26.004455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1185/2373] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:25 2022] DELETE /identity/v3/users/4760be60dd434c6bad44e911d5621f67 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:26.010173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f6d4c-ae82-4106-a4cd-7582f2aad713 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:26.010574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f6d4c-ae82-4106-a4cd-7582f2aad713 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:26.010982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f6d4c-ae82-4106-a4cd-7582f2aad713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:26.054454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d29f6d4c-ae82-4106-a4cd-7582f2aad713 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:26.056115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d29f6d4c-ae82-4106-a4cd-7582f2aad713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:26.148182 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1189/2374] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:26.157785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:26.159353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:26.246775 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:26.307362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0f757-7859-434e-884b-34ec16e692ef admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:26.314152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85391d9-b7d6-4139-ae34-4ce104436866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:26.315352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85391d9-b7d6-4139-ae34-4ce104436866 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:26.315954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85391d9-b7d6-4139-ae34-4ce104436866 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:26.316463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85391d9-b7d6-4139-ae34-4ce104436866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:26.340224 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85391d9-b7d6-4139-ae34-4ce104436866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:26.342314 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85391d9-b7d6-4139-ae34-4ce104436866 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:26.360313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1186/2375] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:26.762938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f6d4c-ae82-4106-a4cd-7582f2aad713 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:26.765101 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d29f6d4c-ae82-4106-a4cd-7582f2aad713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:26.778363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f6d4c-ae82-4106-a4cd-7582f2aad713 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:26.783924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:26.784920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:26.785361 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:26.785667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] PATH_INFO: `/v3/projects/33e87897fc5c47bcba76036c7a60b327` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:26.787509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e87897fc5c47bcba76036c7a60b327)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:26.790047 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:26.860873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e87897fc5c47bcba76036c7a60b327', 'request_id': 'req-423e38a0-b8d3-4357-a3b1-35869ee653cc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:26.875410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e87897fc5c47bcba76036c7a60b327', 'request_id': 'req-423e38a0-b8d3-4357-a3b1-35869ee653cc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:26.882988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e87897fc5c47bcba76036c7a60b327', 'request_id': 'req-423e38a0-b8d3-4357-a3b1-35869ee653cc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:26.901430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] The token cache is being invalidate because project 33e87897fc5c47bcba76036c7a60b327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:26.902112 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:26.903557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-423e38a0-b8d3-4357-a3b1-35869ee653cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:26.905460 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1190/2376] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:26 2022] DELETE /identity/v3/projects/33e87897fc5c47bcba76036c7a60b327 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:26.911333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f9c5d-807c-489d-b735-9be95ae7f8ab None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:26.911782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f9c5d-807c-489d-b735-9be95ae7f8ab None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:26.912458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f9c5d-807c-489d-b735-9be95ae7f8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:26.942459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b98f9c5d-807c-489d-b735-9be95ae7f8ab None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:26.944066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b98f9c5d-807c-489d-b735-9be95ae7f8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:27.055772 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1187/2377] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:27.067884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959ee4f-a90f-4c60-b5ad-b970511d2273 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:27.068178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959ee4f-a90f-4c60-b5ad-b970511d2273 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:27.068456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959ee4f-a90f-4c60-b5ad-b970511d2273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:27.182997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a08b-be69-4eb5-be9d-b2f811a6425b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:27.183293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a08b-be69-4eb5-be9d-b2f811a6425b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:27.183562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a08b-be69-4eb5-be9d-b2f811a6425b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:27.191014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7959ee4f-a90f-4c60-b5ad-b970511d2273 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:27.191616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7959ee4f-a90f-4c60-b5ad-b970511d2273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:27.241211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1191/2378] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:27.245718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959ee4f-a90f-4c60-b5ad-b970511d2273 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:27.253404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3098a08b-be69-4eb5-be9d-b2f811a6425b None None] MFA Rules not processed for user `66366eb7b7044786b23d08676c14670f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:27.254651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3098a08b-be69-4eb5-be9d-b2f811a6425b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:27.272043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959ee4f-a90f-4c60-b5ad-b970511d2273 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:27.308160 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf87468-a2f8-4666-b86e-eb0f4b00f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:27.309938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf87468-a2f8-4666-b86e-eb0f4b00f585 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:27.310226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf87468-a2f8-4666-b86e-eb0f4b00f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:27.310500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf87468-a2f8-4666-b86e-eb0f4b00f585 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:27.313498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf87468-a2f8-4666-b86e-eb0f4b00f585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:27.313837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf87468-a2f8-4666-b86e-eb0f4b00f585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:27.315475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf87468-a2f8-4666-b86e-eb0f4b00f585 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:27.349623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1192/2379] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:38:27.355766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf87468-a2f8-4666-b86e-eb0f4b00f585 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:27.386085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf87468-a2f8-4666-b86e-eb0f4b00f585 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:27.398197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545faa18-8fca-461f-8852-aebf389d63b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:27.398906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545faa18-8fca-461f-8852-aebf389d63b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:27.399346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545faa18-8fca-461f-8852-aebf389d63b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:27.399721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545faa18-8fca-461f-8852-aebf389d63b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:27.400624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545faa18-8fca-461f-8852-aebf389d63b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:27.403747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545faa18-8fca-461f-8852-aebf389d63b8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:27.417440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1188/2380] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:27.437788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:27.480412 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1193/2381] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:27 2022] POST /identity/v3/projects => generated 374 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:38:27.485713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a08b-be69-4eb5-be9d-b2f811a6425b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:27.512981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a08b-be69-4eb5-be9d-b2f811a6425b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:27.677792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb9b6f-3003-4f4d-b1a7-9a03a48d437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:27.678418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb9b6f-3003-4f4d-b1a7-9a03a48d437c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:27.678970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb9b6f-3003-4f4d-b1a7-9a03a48d437c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:27.679260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb9b6f-3003-4f4d-b1a7-9a03a48d437c admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:27.679920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb9b6f-3003-4f4d-b1a7-9a03a48d437c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:27.682150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb9b6f-3003-4f4d-b1a7-9a03a48d437c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:27.720706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1189/2382] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:27 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:27.722037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545faa18-8fca-461f-8852-aebf389d63b8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:27.735048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545faa18-8fca-461f-8852-aebf389d63b8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:27.740227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b182d09-cd5b-4612-b69f-814180189087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:27.740983 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b182d09-cd5b-4612-b69f-814180189087 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:27.741266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b182d09-cd5b-4612-b69f-814180189087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:27.741541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b182d09-cd5b-4612-b69f-814180189087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:27.742234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b182d09-cd5b-4612-b69f-814180189087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:27.744004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b182d09-cd5b-4612-b69f-814180189087 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:27.756944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1194/2383] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:27 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:27.761142 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb9b6f-3003-4f4d-b1a7-9a03a48d437c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:27.769625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:27.776802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb9b6f-3003-4f4d-b1a7-9a03a48d437c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:27.784888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c14bb-1faf-4c81-984f-32827301aff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:27.785641 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c14bb-1faf-4c81-984f-32827301aff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:27.785929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c14bb-1faf-4c81-984f-32827301aff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:27.786798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c14bb-1faf-4c81-984f-32827301aff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:27.787870 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c14bb-1faf-4c81-984f-32827301aff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:27.792047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c14bb-1faf-4c81-984f-32827301aff9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:27.808115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1190/2384] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:27 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:27.813442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b182d09-cd5b-4612-b69f-814180189087 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:27.827705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b182d09-cd5b-4612-b69f-814180189087 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:27.833374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710acf33-9bad-45a1-bab4-e7a0c85b0949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:27.834152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710acf33-9bad-45a1-bab4-e7a0c85b0949 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:27.834436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710acf33-9bad-45a1-bab4-e7a0c85b0949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:27.834713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710acf33-9bad-45a1-bab4-e7a0c85b0949 admin admin] PATH_INFO: `/v3/projects/2190f8e69c254ad28dce603963de77c2/users/9807a015765c477782749da111927b45/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:27.862757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710acf33-9bad-45a1-bab4-e7a0c85b0949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2190f8e69c254ad28dce603963de77c2, user_id=9807a015765c477782749da111927b45, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:27.865957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710acf33-9bad-45a1-bab4-e7a0c85b0949 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:27.912687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1195/2385] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:27 2022] PUT /identity/v3/projects/2190f8e69c254ad28dce603963de77c2/users/9807a015765c477782749da111927b45/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:27.923462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0cb5e-e5cd-4886-86dd-062197e94385 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:27.923754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0cb5e-e5cd-4886-86dd-062197e94385 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:27.924034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0cb5e-e5cd-4886-86dd-062197e94385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:27.977714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b6a0cb5e-e5cd-4886-86dd-062197e94385 None None] MFA Rules not processed for user `9807a015765c477782749da111927b45`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:27.979016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b6a0cb5e-e5cd-4886-86dd-062197e94385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:28.028740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b49e-d5bc-470a-a0a4-4a116905bb76 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:28.030367 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b49e-d5bc-470a-a0a4-4a116905bb76 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:28.031292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b49e-d5bc-470a-a0a4-4a116905bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:28.064395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1191/2386] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:28.077347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-caa2b49e-d5bc-470a-a0a4-4a116905bb76 None None] MFA Rules not processed for user `d3848643d2c44951b78814ccf7d504af`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:28.078015 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0cb5e-e5cd-4886-86dd-062197e94385 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:28.078921 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-caa2b49e-d5bc-470a-a0a4-4a116905bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:28.079450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b6a0cb5e-e5cd-4886-86dd-062197e94385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:28.172378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0cb5e-e5cd-4886-86dd-062197e94385 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:28.177728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b4a70-1352-4c6e-9b13-6c7f40baf3e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:28.178649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b4a70-1352-4c6e-9b13-6c7f40baf3e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:28.179213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b4a70-1352-4c6e-9b13-6c7f40baf3e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:28.179727 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b4a70-1352-4c6e-9b13-6c7f40baf3e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:28.185744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1196/2387] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:28.191450 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b4a70-1352-4c6e-9b13-6c7f40baf3e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:28.193514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b4a70-1352-4c6e-9b13-6c7f40baf3e2 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:28.206168 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1192/2388] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:38:28.244810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b49e-d5bc-470a-a0a4-4a116905bb76 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:28.321815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b49e-d5bc-470a-a0a4-4a116905bb76 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:28.326645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:28.327303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:28.327583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:28.327861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] PATH_INFO: `/v3/users/9807a015765c477782749da111927b45` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:28.329021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9807a015765c477782749da111927b45)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:28.331025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:28.392038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9807a015765c477782749da111927b45'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:28.393416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:28.408930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9807a015765c477782749da111927b45'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:28.418595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9807a015765c477782749da111927b45'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:28.428042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9807a015765c477782749da111927b45'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:28.428979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9807a015765c477782749da111927b45'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:28.436803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1197/2389] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:28 2022] DELETE /identity/v3/users/9807a015765c477782749da111927b45 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:28.441206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7414b68-cbf6-426f-a974-a72ab1ff5320 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:28.441496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7414b68-cbf6-426f-a974-a72ab1ff5320 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:28.441797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7414b68-cbf6-426f-a974-a72ab1ff5320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:28.466999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a7414b68-cbf6-426f-a974-a72ab1ff5320 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:28.468315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a7414b68-cbf6-426f-a974-a72ab1ff5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:28.538752 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1193/2390] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:28.552513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:28.554253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:28.613612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7414b68-cbf6-426f-a974-a72ab1ff5320 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:28.615616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a7414b68-cbf6-426f-a974-a72ab1ff5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:28.642534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5ad3c-5547-45dd-973f-3a3a729f9024 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:28.647259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969f07b-0084-4e5c-9bce-d4284430cd92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:28.648225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969f07b-0084-4e5c-9bce-d4284430cd92 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:28.648577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969f07b-0084-4e5c-9bce-d4284430cd92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:28.649672 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969f07b-0084-4e5c-9bce-d4284430cd92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:28.662793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:28.681712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969f07b-0084-4e5c-9bce-d4284430cd92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:28.693531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969f07b-0084-4e5c-9bce-d4284430cd92 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:28.730829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1198/2391] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:28.813634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7414b68-cbf6-426f-a974-a72ab1ff5320 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:28.821890 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:28.823330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:28.823579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:28.824117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:28.827189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:28.937482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:28.939165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:28.960832 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1194/2392] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:29.172380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969f07b-0084-4e5c-9bce-d4284430cd92 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:29.174085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7969f07b-0084-4e5c-9bce-d4284430cd92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:29.199134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969f07b-0084-4e5c-9bce-d4284430cd92 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:29.243078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:29.248685 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:29.249319 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:29.250634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:29.250855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:29.252791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] PATH_INFO: `/v3/users/387c3f2fc4e04c1ca7231a35ab4b0eff` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:29.256260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=387c3f2fc4e04c1ca7231a35ab4b0eff)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:29.262583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:29.294964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dffcc7-19ac-40fb-9a0f-e7f88c648b39 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:29.300465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:29.301500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:29.301972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:29.302487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] PATH_INFO: `/v3/projects/2190f8e69c254ad28dce603963de77c2` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:29.304396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2190f8e69c254ad28dce603963de77c2)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:29.306979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:29.396509 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '387c3f2fc4e04c1ca7231a35ab4b0eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.398127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2190f8e69c254ad28dce603963de77c2', 'request_id': 'req-63279874-f7bb-4434-96e9-8c42e91b5212'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.409432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2190f8e69c254ad28dce603963de77c2', 'request_id': 'req-63279874-f7bb-4434-96e9-8c42e91b5212'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.424696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2190f8e69c254ad28dce603963de77c2', 'request_id': 'req-63279874-f7bb-4434-96e9-8c42e91b5212'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.437295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '387c3f2fc4e04c1ca7231a35ab4b0eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.449723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] The token cache is being invalidate because project 2190f8e69c254ad28dce603963de77c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:29.449723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.449975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '387c3f2fc4e04c1ca7231a35ab4b0eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.450475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63279874-f7bb-4434-96e9-8c42e91b5212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.457710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1195/2393] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:29 2022] DELETE /identity/v3/projects/2190f8e69c254ad28dce603963de77c2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:29.472369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '387c3f2fc4e04c1ca7231a35ab4b0eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.473704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '387c3f2fc4e04c1ca7231a35ab4b0eff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:29.491178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1199/2394] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:29 2022] DELETE /identity/v3/users/387c3f2fc4e04c1ca7231a35ab4b0eff => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:29.495807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769e186-ec73-4e68-b658-79a3541cbac1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:29.496773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769e186-ec73-4e68-b658-79a3541cbac1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:29.497145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769e186-ec73-4e68-b658-79a3541cbac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:29.562460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e769e186-ec73-4e68-b658-79a3541cbac1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:29.570582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e769e186-ec73-4e68-b658-79a3541cbac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:29.694688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:29.735379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1196/2395] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:29.748283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:29.749867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:29.815975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:29.838410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f748be0-3a87-44ce-9964-2411fb4f5664 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:29.845454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148d19e-121b-470f-9b68-f3ecbcb68097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:29.846262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148d19e-121b-470f-9b68-f3ecbcb68097 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:29.846623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148d19e-121b-470f-9b68-f3ecbcb68097 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:29.846984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148d19e-121b-470f-9b68-f3ecbcb68097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:29.859368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148d19e-121b-470f-9b68-f3ecbcb68097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:29.861000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148d19e-121b-470f-9b68-f3ecbcb68097 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:29.874478 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1200/2396] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:30.506356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769e186-ec73-4e68-b658-79a3541cbac1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:30.507849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e769e186-ec73-4e68-b658-79a3541cbac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:30.533975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769e186-ec73-4e68-b658-79a3541cbac1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:30.545936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:30.546598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:30.546882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:30.547155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] PATH_INFO: `/v3/projects/0209b8de4b2b43f886ef0c298c46cd3a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:30.548396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0209b8de4b2b43f886ef0c298c46cd3a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:30.550362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:30.678451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0209b8de4b2b43f886ef0c298c46cd3a', 'request_id': 'req-e8cdbbf8-39fb-46a1-b622-b68788a7e939'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:30.686329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0209b8de4b2b43f886ef0c298c46cd3a', 'request_id': 'req-e8cdbbf8-39fb-46a1-b622-b68788a7e939'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:30.692872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0209b8de4b2b43f886ef0c298c46cd3a', 'request_id': 'req-e8cdbbf8-39fb-46a1-b622-b68788a7e939'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:30.708683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] The token cache is being invalidate because project 0209b8de4b2b43f886ef0c298c46cd3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:30.708951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:30.709596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e8cdbbf8-39fb-46a1-b622-b68788a7e939 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:30.712390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1197/2397] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:30 2022] DELETE /identity/v3/projects/0209b8de4b2b43f886ef0c298c46cd3a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:30.717076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7e799-a104-427b-8559-c549360b062d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:30.717635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7e799-a104-427b-8559-c549360b062d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:30.717749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7e799-a104-427b-8559-c549360b062d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:30.749644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-62d7e799-a104-427b-8559-c549360b062d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:30.752904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-62d7e799-a104-427b-8559-c549360b062d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:30.860665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1201/2398] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:30.870088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fcac2-e3aa-4f49-b86f-a6011a6c8698 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:30.870392 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fcac2-e3aa-4f49-b86f-a6011a6c8698 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:30.870661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fcac2-e3aa-4f49-b86f-a6011a6c8698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:30.927657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ec1fcac2-e3aa-4f49-b86f-a6011a6c8698 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:30.929359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ec1fcac2-e3aa-4f49-b86f-a6011a6c8698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:30.958321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1198/2399] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:30.959526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7e799-a104-427b-8559-c549360b062d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:30.977658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7e799-a104-427b-8559-c549360b062d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:30.989596 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58329d-645d-44aa-80a2-4c9ad489df9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:30.990561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58329d-645d-44aa-80a2-4c9ad489df9c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:30.991492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58329d-645d-44aa-80a2-4c9ad489df9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:30.991918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58329d-645d-44aa-80a2-4c9ad489df9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:30.993726 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58329d-645d-44aa-80a2-4c9ad489df9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:30.994173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58329d-645d-44aa-80a2-4c9ad489df9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:30.997448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58329d-645d-44aa-80a2-4c9ad489df9c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:31.018858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1202/2400] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:38:31.023430 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fcac2-e3aa-4f49-b86f-a6011a6c8698 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:31.036238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fcac2-e3aa-4f49-b86f-a6011a6c8698 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:31.043328 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bba0c-a001-48b9-b2dd-c33ad4ccdba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:31.043997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bba0c-a001-48b9-b2dd-c33ad4ccdba7 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:31.044417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bba0c-a001-48b9-b2dd-c33ad4ccdba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:31.044766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bba0c-a001-48b9-b2dd-c33ad4ccdba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:31.045957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bba0c-a001-48b9-b2dd-c33ad4ccdba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:31.048801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bba0c-a001-48b9-b2dd-c33ad4ccdba7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:31.094634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1199/2401] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:31 2022] POST /identity/v3/projects => generated 372 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:31.095413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58329d-645d-44aa-80a2-4c9ad489df9c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:31.106531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58329d-645d-44aa-80a2-4c9ad489df9c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:31.204660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f9b98-ec73-461e-b83a-d061b7c17bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:31.205608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f9b98-ec73-461e-b83a-d061b7c17bea admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:31.205919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f9b98-ec73-461e-b83a-d061b7c17bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:31.206200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f9b98-ec73-461e-b83a-d061b7c17bea admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:31.206879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f9b98-ec73-461e-b83a-d061b7c17bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:31.208993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f9b98-ec73-461e-b83a-d061b7c17bea admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:31.247515 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1203/2402] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:31 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:38:31.252196 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bba0c-a001-48b9-b2dd-c33ad4ccdba7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:31.266572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bba0c-a001-48b9-b2dd-c33ad4ccdba7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:31.275769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c616428-d967-450d-9a40-dca719ae6f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:31.276881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c616428-d967-450d-9a40-dca719ae6f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:31.277375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c616428-d967-450d-9a40-dca719ae6f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:31.278226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c616428-d967-450d-9a40-dca719ae6f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:31.279102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c616428-d967-450d-9a40-dca719ae6f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:31.281030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c616428-d967-450d-9a40-dca719ae6f14 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:31.295898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1200/2403] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:31 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:38:31.299811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f9b98-ec73-461e-b83a-d061b7c17bea admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:31.318299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f9b98-ec73-461e-b83a-d061b7c17bea admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:31.330548 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69e923-9581-461a-bd12-65a918928e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:31.331311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69e923-9581-461a-bd12-65a918928e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:31.331667 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69e923-9581-461a-bd12-65a918928e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:31.332028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69e923-9581-461a-bd12-65a918928e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:31.333195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69e923-9581-461a-bd12-65a918928e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:31.338357 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69e923-9581-461a-bd12-65a918928e98 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:31.358629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1204/2404] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:31 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:31.366062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c616428-d967-450d-9a40-dca719ae6f14 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:31.389686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c616428-d967-450d-9a40-dca719ae6f14 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:31.395611 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596eca6a-5712-4066-b226-3762a4307e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:31.397282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596eca6a-5712-4066-b226-3762a4307e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:31.397805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596eca6a-5712-4066-b226-3762a4307e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:31.398366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596eca6a-5712-4066-b226-3762a4307e74 admin admin] PATH_INFO: `/v3/projects/b3f3bf24fbbd42919ad87cc04e4f6042/users/289058107592455fbf5af315fba7a126/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:31.418938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596eca6a-5712-4066-b226-3762a4307e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f3bf24fbbd42919ad87cc04e4f6042, user_id=289058107592455fbf5af315fba7a126, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:31.421782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596eca6a-5712-4066-b226-3762a4307e74 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:31.443322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1201/2405] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:31 2022] PUT /identity/v3/projects/b3f3bf24fbbd42919ad87cc04e4f6042/users/289058107592455fbf5af315fba7a126/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:31.450246 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb69e07-d791-4587-80f8-7378ab1bb47d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:31.450565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb69e07-d791-4587-80f8-7378ab1bb47d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:31.450813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb69e07-d791-4587-80f8-7378ab1bb47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:31.488025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3cb69e07-d791-4587-80f8-7378ab1bb47d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:31.489673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3cb69e07-d791-4587-80f8-7378ab1bb47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:31.542558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:31.578207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1205/2406] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:31.589725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596eca6a-5712-4066-b226-3762a4307e74 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:31.591782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-596eca6a-5712-4066-b226-3762a4307e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:31.698123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596eca6a-5712-4066-b226-3762a4307e74 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:31.703284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a3b6c-1a66-4c46-9311-c287dc3394ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:31.704104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3b6c-1a66-4c46-9311-c287dc3394ad service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:31.704386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3b6c-1a66-4c46-9311-c287dc3394ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:31.704655 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3b6c-1a66-4c46-9311-c287dc3394ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:31.729717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a3b6c-1a66-4c46-9311-c287dc3394ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:31.733371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a3b6c-1a66-4c46-9311-c287dc3394ad service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:31.750780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1202/2407] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:31.952820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cb971-8ad8-4c00-9fb6-fee3376991d3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:31.953403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cb971-8ad8-4c00-9fb6-fee3376991d3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:31.953403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cb971-8ad8-4c00-9fb6-fee3376991d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:31.988534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c18cb971-8ad8-4c00-9fb6-fee3376991d3 None None] MFA Rules not processed for user `d3848643d2c44951b78814ccf7d504af`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:31.990469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c18cb971-8ad8-4c00-9fb6-fee3376991d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:32.095488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1206/2408] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:36.044190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4dffc-6236-44f8-95fd-47e4f70373f6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:36.045116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4dffc-6236-44f8-95fd-47e4f70373f6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:36.045397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4dffc-6236-44f8-95fd-47e4f70373f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:36.073446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1db4dffc-6236-44f8-95fd-47e4f70373f6 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:36.075017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1db4dffc-6236-44f8-95fd-47e4f70373f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:36.081837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:36.183984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1203/2409] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:38:36 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:36.253628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cb971-8ad8-4c00-9fb6-fee3376991d3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:36.256158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c18cb971-8ad8-4c00-9fb6-fee3376991d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:36.366773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cb971-8ad8-4c00-9fb6-fee3376991d3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:36.379216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fd612-4291-4118-ae44-58e8e748a7bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:36.380241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fd612-4291-4118-ae44-58e8e748a7bc service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:36.380583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fd612-4291-4118-ae44-58e8e748a7bc service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:36.381019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fd612-4291-4118-ae44-58e8e748a7bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:36.402186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fd612-4291-4118-ae44-58e8e748a7bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:36.404369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fd612-4291-4118-ae44-58e8e748a7bc service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:36.421713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1207/2410] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:38:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:38:37.642918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e1004-56aa-43c3-bb14-1a1d228aa5d6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:37.643442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e1004-56aa-43c3-bb14-1a1d228aa5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:37.643442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e1004-56aa-43c3-bb14-1a1d228aa5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:37.687420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dc7e1004-56aa-43c3-bb14-1a1d228aa5d6 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:37.689220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dc7e1004-56aa-43c3-bb14-1a1d228aa5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:37.709709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1204/2411] 10.222.0.100 () {58 vars in 972 bytes} [Thu Sep 1 17:38:37 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:37.718009 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fd612-4291-4118-ae44-58e8e748a7bc service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:37.735643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fd612-4291-4118-ae44-58e8e748a7bc service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:37.747121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ffd6a-c384-45a3-a1e5-d00fc6e44171 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:37.748011 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ffd6a-c384-45a3-a1e5-d00fc6e44171 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:37.748632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ffd6a-c384-45a3-a1e5-d00fc6e44171 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:37.749253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ffd6a-c384-45a3-a1e5-d00fc6e44171 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:37.766551 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ffd6a-c384-45a3-a1e5-d00fc6e44171 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:37.771804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ffd6a-c384-45a3-a1e5-d00fc6e44171 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:37.786532 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1208/2412] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:38:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:38:38.150514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbc230-f265-46de-93c5-57e3d212cd80 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:38.150794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbc230-f265-46de-93c5-57e3d212cd80 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:38.151460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbc230-f265-46de-93c5-57e3d212cd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:38.174528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-39fbc230-f265-46de-93c5-57e3d212cd80 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:38.176017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-39fbc230-f265-46de-93c5-57e3d212cd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:38.204425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1205/2413] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:38:38 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:38.215330 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ffd6a-c384-45a3-a1e5-d00fc6e44171 service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:38.229773 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ffd6a-c384-45a3-a1e5-d00fc6e44171 service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:38.237802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe091f7c-7aff-4070-8c10-8583c261c824 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:38.239218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe091f7c-7aff-4070-8c10-8583c261c824 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:38.239218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe091f7c-7aff-4070-8c10-8583c261c824 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:38.239218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe091f7c-7aff-4070-8c10-8583c261c824 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:38.251243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe091f7c-7aff-4070-8c10-8583c261c824 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:38.253220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe091f7c-7aff-4070-8c10-8583c261c824 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:38.273367 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1209/2414] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:38:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:38:39.160251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9007-9826-49cc-b285-433c3600e8c6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:39.161159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9007-9826-49cc-b285-433c3600e8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:39.161437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9007-9826-49cc-b285-433c3600e8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:39.185307 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-74ec9007-9826-49cc-b285-433c3600e8c6 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:39.186857 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-74ec9007-9826-49cc-b285-433c3600e8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:39.228171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1206/2415] 10.222.0.100 () {58 vars in 972 bytes} [Thu Sep 1 17:38:39 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:39.235095 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe091f7c-7aff-4070-8c10-8583c261c824 service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:39.259115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe091f7c-7aff-4070-8c10-8583c261c824 service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:39.279454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c446c43-6125-484f-9663-60186859bfe7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:39.280274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c446c43-6125-484f-9663-60186859bfe7 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:39.280844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c446c43-6125-484f-9663-60186859bfe7 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:39.281483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c446c43-6125-484f-9663-60186859bfe7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:39.303107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c446c43-6125-484f-9663-60186859bfe7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:39.307218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c446c43-6125-484f-9663-60186859bfe7 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:39.330247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1210/2416] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:38:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:38:41.152257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9026-9339-4591-8509-53824b55ca57 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:41.153241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9026-9339-4591-8509-53824b55ca57 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:41.155154 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9026-9339-4591-8509-53824b55ca57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:41.234016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e1ba9026-9339-4591-8509-53824b55ca57 None None] MFA Rules not processed for user `289058107592455fbf5af315fba7a126`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:41.238402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e1ba9026-9339-4591-8509-53824b55ca57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:41.456206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1207/2417] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:41.457683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c446c43-6125-484f-9663-60186859bfe7 service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:41.473137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c446c43-6125-484f-9663-60186859bfe7 service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:41.502655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1917ab-1d5d-430a-801f-8880bfa37634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:41.504925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1917ab-1d5d-430a-801f-8880bfa37634 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:41.506547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1917ab-1d5d-430a-801f-8880bfa37634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:41.507108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1917ab-1d5d-430a-801f-8880bfa37634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:41.509222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1917ab-1d5d-430a-801f-8880bfa37634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:41.513966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1917ab-1d5d-430a-801f-8880bfa37634 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:41.590090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1211/2418] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:41 2022] POST /identity/v3/projects => generated 374 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:41.594286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9026-9339-4591-8509-53824b55ca57 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:41.657425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9026-9339-4591-8509-53824b55ca57 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:41.820684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9ffe2-caef-4333-9f8f-c98a274fba12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:41.822750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9ffe2-caef-4333-9f8f-c98a274fba12 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:41.823646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9ffe2-caef-4333-9f8f-c98a274fba12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:41.824389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9ffe2-caef-4333-9f8f-c98a274fba12 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:41.825553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9ffe2-caef-4333-9f8f-c98a274fba12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:41.828097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9ffe2-caef-4333-9f8f-c98a274fba12 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:41.903976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1208/2419] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:41 2022] POST /identity/v3/users => generated 294 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:41.908393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1917ab-1d5d-430a-801f-8880bfa37634 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:41.931284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1917ab-1d5d-430a-801f-8880bfa37634 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:41.949313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7739fe-9b4e-4d44-b7c9-f9f9850d77fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:41.949965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7739fe-9b4e-4d44-b7c9-f9f9850d77fd admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:41.950240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7739fe-9b4e-4d44-b7c9-f9f9850d77fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:41.950505 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7739fe-9b4e-4d44-b7c9-f9f9850d77fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:41.951235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7739fe-9b4e-4d44-b7c9-f9f9850d77fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:41.954284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7739fe-9b4e-4d44-b7c9-f9f9850d77fd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:41.983734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1212/2420] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:41 2022] GET /identity/v3/roles => generated 1427 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:41.988627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9ffe2-caef-4333-9f8f-c98a274fba12 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:42.000105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9ffe2-caef-4333-9f8f-c98a274fba12 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:42.005182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad921cc4-2409-429a-86d4-707ee49a7774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:42.006246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad921cc4-2409-429a-86d4-707ee49a7774 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:42.006546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad921cc4-2409-429a-86d4-707ee49a7774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:42.007480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad921cc4-2409-429a-86d4-707ee49a7774 admin admin] PATH_INFO: `/v3/projects/0930f174a68749ad9865046cdf512fa4/users/ea59a76ca4794d3084495a5e20bb1517/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:42.022546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:42.031511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad921cc4-2409-429a-86d4-707ee49a7774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0930f174a68749ad9865046cdf512fa4, user_id=ea59a76ca4794d3084495a5e20bb1517, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:42.035094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad921cc4-2409-429a-86d4-707ee49a7774 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:42.062491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1209/2421] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:41 2022] PUT /identity/v3/projects/0930f174a68749ad9865046cdf512fa4/users/ea59a76ca4794d3084495a5e20bb1517/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:42.066960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7739fe-9b4e-4d44-b7c9-f9f9850d77fd admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:42.081033 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7739fe-9b4e-4d44-b7c9-f9f9850d77fd admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:42.093482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:42.204452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f6d1a-b4fc-4a91-94aa-edcd135b48a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:42.205098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f6d1a-b4fc-4a91-94aa-edcd135b48a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:42.205605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f6d1a-b4fc-4a91-94aa-edcd135b48a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:42.206034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f6d1a-b4fc-4a91-94aa-edcd135b48a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:42.206682 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f6d1a-b4fc-4a91-94aa-edcd135b48a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:42.208301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f6d1a-b4fc-4a91-94aa-edcd135b48a0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:42.221084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1213/2422] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:42 2022] GET /identity/v3/roles => generated 1427 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:42.225208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad921cc4-2409-429a-86d4-707ee49a7774 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:42.237404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad921cc4-2409-429a-86d4-707ee49a7774 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:42.243747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b451a-b0c7-46cd-ac94-ff3ef7d639d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:42.244785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b451a-b0c7-46cd-ac94-ff3ef7d639d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:42.245217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b451a-b0c7-46cd-ac94-ff3ef7d639d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:42.245749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b451a-b0c7-46cd-ac94-ff3ef7d639d2 admin admin] PATH_INFO: `/v3/projects/0930f174a68749ad9865046cdf512fa4/users/ea59a76ca4794d3084495a5e20bb1517/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:42.247778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b451a-b0c7-46cd-ac94-ff3ef7d639d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0930f174a68749ad9865046cdf512fa4, user_id=ea59a76ca4794d3084495a5e20bb1517, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:42.251517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b451a-b0c7-46cd-ac94-ff3ef7d639d2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:42.264581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1210/2423] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:42 2022] PUT /identity/v3/projects/0930f174a68749ad9865046cdf512fa4/users/ea59a76ca4794d3084495a5e20bb1517/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:38:42.472377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f6d1a-b4fc-4a91-94aa-edcd135b48a0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:42.475334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-267f6d1a-b4fc-4a91-94aa-edcd135b48a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:42.501761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f6d1a-b4fc-4a91-94aa-edcd135b48a0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:42.641367 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:42.642366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:42.642786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:42.644931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] PATH_INFO: `/v3/users/c2e0a450471043678b080515896d0f50` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:42.652304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e0a450471043678b080515896d0f50)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:42.655205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:42.784067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e0a450471043678b080515896d0f50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:42.812006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e0a450471043678b080515896d0f50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:42.825656 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e0a450471043678b080515896d0f50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:42.838780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e0a450471043678b080515896d0f50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:42.840280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e0a450471043678b080515896d0f50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:42.852685 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1214/2424] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:38:42 2022] DELETE /identity/v3/users/c2e0a450471043678b080515896d0f50 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:42.854325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207adb-c441-4909-af55-b08c78490843 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:42.854556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207adb-c441-4909-af55-b08c78490843 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:42.854839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207adb-c441-4909-af55-b08c78490843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:42.886086 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2b207adb-c441-4909-af55-b08c78490843 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:42.887553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2b207adb-c441-4909-af55-b08c78490843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:42.977564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1211/2425] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:42.988875 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:42.991656 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:43.039815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:43.092791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94860db0-aacd-4711-a5ce-298022b2a011 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:43.098461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d02f8-7d36-40a5-8aad-0deb396b4ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:43.099633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d02f8-7d36-40a5-8aad-0deb396b4ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:43.100230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d02f8-7d36-40a5-8aad-0deb396b4ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:43.100690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d02f8-7d36-40a5-8aad-0deb396b4ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:43.113621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d02f8-7d36-40a5-8aad-0deb396b4ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:43.116162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d02f8-7d36-40a5-8aad-0deb396b4ea6 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:43.127390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1215/2426] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:43.356823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b207adb-c441-4909-af55-b08c78490843 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:43.358380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2b207adb-c441-4909-af55-b08c78490843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:43.385993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b207adb-c441-4909-af55-b08c78490843 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:43.395334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55358325-6062-4df4-8135-89787039460e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:43.395986 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55358325-6062-4df4-8135-89787039460e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:43.396268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55358325-6062-4df4-8135-89787039460e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:43.396542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55358325-6062-4df4-8135-89787039460e admin admin] PATH_INFO: `/v3/projects/8643f179145e4e1aba7994ada9821a30` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:43.404323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55358325-6062-4df4-8135-89787039460e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8643f179145e4e1aba7994ada9821a30)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:43.406543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55358325-6062-4df4-8135-89787039460e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:43.524417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55358325-6062-4df4-8135-89787039460e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8643f179145e4e1aba7994ada9821a30', 'request_id': 'req-55358325-6062-4df4-8135-89787039460e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:43.535233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55358325-6062-4df4-8135-89787039460e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8643f179145e4e1aba7994ada9821a30', 'request_id': 'req-55358325-6062-4df4-8135-89787039460e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:43.545353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55358325-6062-4df4-8135-89787039460e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8643f179145e4e1aba7994ada9821a30', 'request_id': 'req-55358325-6062-4df4-8135-89787039460e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:43.569621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55358325-6062-4df4-8135-89787039460e admin admin] The token cache is being invalidate because project 8643f179145e4e1aba7994ada9821a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:38:43.570183 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55358325-6062-4df4-8135-89787039460e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:43.571018 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-55358325-6062-4df4-8135-89787039460e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:38:43.573612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1212/2427] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:38:43 2022] DELETE /identity/v3/projects/8643f179145e4e1aba7994ada9821a30 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:43.577919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aed4f8-f24a-4a31-aee9-aaae67fadf12 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:43.578420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aed4f8-f24a-4a31-aee9-aaae67fadf12 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:43.578528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aed4f8-f24a-4a31-aee9-aaae67fadf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:43.598892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-78aed4f8-f24a-4a31-aee9-aaae67fadf12 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:43.600126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-78aed4f8-f24a-4a31-aee9-aaae67fadf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:43.685789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1216/2428] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:43.692868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35e36b-dc98-415d-9fe6-8fd06dd943de None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:43.693428 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35e36b-dc98-415d-9fe6-8fd06dd943de None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:43.694030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35e36b-dc98-415d-9fe6-8fd06dd943de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:43.724800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9d35e36b-dc98-415d-9fe6-8fd06dd943de None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:43.726063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9d35e36b-dc98-415d-9fe6-8fd06dd943de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:43.736858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1213/2429] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:38:43.741356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aed4f8-f24a-4a31-aee9-aaae67fadf12 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:43.755717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aed4f8-f24a-4a31-aee9-aaae67fadf12 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:43.764225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdd6c0-5135-46a2-9db1-1c00346b1e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:43.764866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd6c0-5135-46a2-9db1-1c00346b1e51 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:43.765145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd6c0-5135-46a2-9db1-1c00346b1e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:43.765416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd6c0-5135-46a2-9db1-1c00346b1e51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:43.767198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdd6c0-5135-46a2-9db1-1c00346b1e51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:38:43.767198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdd6c0-5135-46a2-9db1-1c00346b1e51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:43.805894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdd6c0-5135-46a2-9db1-1c00346b1e51 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:43.875616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1217/2430] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:38:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:38:43.879329 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35e36b-dc98-415d-9fe6-8fd06dd943de None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:43.900837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35e36b-dc98-415d-9fe6-8fd06dd943de None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:43.915024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adccdd-b2c6-4e5a-b1d1-f772f5bd4df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:43.915686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adccdd-b2c6-4e5a-b1d1-f772f5bd4df9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:43.918952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adccdd-b2c6-4e5a-b1d1-f772f5bd4df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:43.919288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adccdd-b2c6-4e5a-b1d1-f772f5bd4df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:43.921095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adccdd-b2c6-4e5a-b1d1-f772f5bd4df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:43.925412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adccdd-b2c6-4e5a-b1d1-f772f5bd4df9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:44.005718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1214/2431] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:38:43 2022] POST /identity/v3/projects => generated 390 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:44.008389 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdd6c0-5135-46a2-9db1-1c00346b1e51 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:44.026829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdd6c0-5135-46a2-9db1-1c00346b1e51 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:44.140026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5587b09-fa79-421b-b4d4-fe0c7944df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:44.140768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5587b09-fa79-421b-b4d4-fe0c7944df0b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:44.141050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5587b09-fa79-421b-b4d4-fe0c7944df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:44.141322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5587b09-fa79-421b-b4d4-fe0c7944df0b admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:44.145408 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5587b09-fa79-421b-b4d4-fe0c7944df0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:44.150291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5587b09-fa79-421b-b4d4-fe0c7944df0b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:44.209452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1218/2432] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:38:44 2022] POST /identity/v3/users => generated 296 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:38:44.213871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adccdd-b2c6-4e5a-b1d1-f772f5bd4df9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:44.230925 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adccdd-b2c6-4e5a-b1d1-f772f5bd4df9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:44.258939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e45f2-e9e7-4d63-b0b5-12f09bcf9943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:44.261378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e45f2-e9e7-4d63-b0b5-12f09bcf9943 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:44.261378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e45f2-e9e7-4d63-b0b5-12f09bcf9943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:44.261378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e45f2-e9e7-4d63-b0b5-12f09bcf9943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:44.262452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e45f2-e9e7-4d63-b0b5-12f09bcf9943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:44.265911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e45f2-e9e7-4d63-b0b5-12f09bcf9943 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:44.295397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1215/2433] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:44 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:44.299215 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5587b09-fa79-421b-b4d4-fe0c7944df0b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:44.314642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5587b09-fa79-421b-b4d4-fe0c7944df0b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:44.334647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26e178-35e4-4550-aae3-a301f0cb2858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:44.335265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26e178-35e4-4550-aae3-a301f0cb2858 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:44.335541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26e178-35e4-4550-aae3-a301f0cb2858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:44.335999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26e178-35e4-4550-aae3-a301f0cb2858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:44.337007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26e178-35e4-4550-aae3-a301f0cb2858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:44.338648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26e178-35e4-4550-aae3-a301f0cb2858 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:44.372768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1219/2434] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:38:44 2022] GET /identity/v3/roles => generated 1427 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:38:44.377116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e45f2-e9e7-4d63-b0b5-12f09bcf9943 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:44.407863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e45f2-e9e7-4d63-b0b5-12f09bcf9943 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:44.428236 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ed08c-cc79-4788-9e43-3964847f74c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:44.428862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ed08c-cc79-4788-9e43-3964847f74c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:44.430217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ed08c-cc79-4788-9e43-3964847f74c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:44.431121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ed08c-cc79-4788-9e43-3964847f74c7 admin admin] PATH_INFO: `/v3/projects/a4720f52f41641ed92b785e911a4e0ab/users/5c3d25a4883448d9bedfee863d0d5c66/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:44.459369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ed08c-cc79-4788-9e43-3964847f74c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4720f52f41641ed92b785e911a4e0ab, user_id=5c3d25a4883448d9bedfee863d0d5c66, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:44.463924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ed08c-cc79-4788-9e43-3964847f74c7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:44.469064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:44.490685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1216/2435] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:38:44 2022] PUT /identity/v3/projects/a4720f52f41641ed92b785e911a4e0ab/users/5c3d25a4883448d9bedfee863d0d5c66/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:38:44.590897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4365507-2ce7-42cf-a607-8e13097ed814 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:44.591481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4365507-2ce7-42cf-a607-8e13097ed814 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:44.591591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4365507-2ce7-42cf-a607-8e13097ed814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:44.682629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e4365507-2ce7-42cf-a607-8e13097ed814 None None] MFA Rules not processed for user `5c3d25a4883448d9bedfee863d0d5c66`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:44.683863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e4365507-2ce7-42cf-a607-8e13097ed814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:44.753745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:44.875085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1220/2436] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:44 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:44.886843 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ed08c-cc79-4788-9e43-3964847f74c7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:44.888849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-793ed08c-cc79-4788-9e43-3964847f74c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:45.071933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ed08c-cc79-4788-9e43-3964847f74c7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:45.076571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233e1ba-2bbb-4924-9a84-c2a9d60623a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:45.077294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233e1ba-2bbb-4924-9a84-c2a9d60623a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:45.077619 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233e1ba-2bbb-4924-9a84-c2a9d60623a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:45.078062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233e1ba-2bbb-4924-9a84-c2a9d60623a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:45.097791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233e1ba-2bbb-4924-9a84-c2a9d60623a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:45.101183 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233e1ba-2bbb-4924-9a84-c2a9d60623a6 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:45.123560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1217/2437] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:44 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:38:49.482192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee73864-5915-44c2-b571-b901c3a918e0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:49.483643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee73864-5915-44c2-b571-b901c3a918e0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:49.483962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee73864-5915-44c2-b571-b901c3a918e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:49.519362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-eee73864-5915-44c2-b571-b901c3a918e0 None None] MFA Rules not processed for user `ea59a76ca4794d3084495a5e20bb1517`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:49.520929 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eee73864-5915-44c2-b571-b901c3a918e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:49.611572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2cb81-9ac7-4b87-a8f6-d0fd09c94520 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:49.612023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2cb81-9ac7-4b87-a8f6-d0fd09c94520 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:49.612260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2cb81-9ac7-4b87-a8f6-d0fd09c94520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:49.633993 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1221/2438] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:49.655280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee73864-5915-44c2-b571-b901c3a918e0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:38:49.657187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eee73864-5915-44c2-b571-b901c3a918e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:49.666043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-78a2cb81-9ac7-4b87-a8f6-d0fd09c94520 None None] MFA Rules not processed for user `5647214c5b3f438eb3ef1ddc14153d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:49.669671 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-78a2cb81-9ac7-4b87-a8f6-d0fd09c94520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:49.752334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1218/2439] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:49 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:49.762627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee73864-5915-44c2-b571-b901c3a918e0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:38:49.770676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a5b0d-1435-427d-8ce1-1cfdb6560da4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:38:49.772208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a5b0d-1435-427d-8ce1-1cfdb6560da4 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:49.772670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a5b0d-1435-427d-8ce1-1cfdb6560da4 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:49.773020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a5b0d-1435-427d-8ce1-1cfdb6560da4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:49.777007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e62a5b0d-1435-427d-8ce1-1cfdb6560da4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:49.830982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:38:49.845111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a5b0d-1435-427d-8ce1-1cfdb6560da4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:38:49.846917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a5b0d-1435-427d-8ce1-1cfdb6560da4 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:38:49.860357 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1222/2440] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:38:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:38:51.990060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c51b04-07c9-4688-9233-47e19c34770a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:38:51.990579 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c51b04-07c9-4688-9233-47e19c34770a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:38:51.990692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c51b04-07c9-4688-9233-47e19c34770a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:38:52.011385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b7c51b04-07c9-4688-9233-47e19c34770a None None] MFA Rules not processed for user `289058107592455fbf5af315fba7a126`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:38:52.012973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b7c51b04-07c9-4688-9233-47e19c34770a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:38:52.023717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1219/2441] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:38:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:38:59.853455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:02.005244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=56 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:03.558531 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b561a-015b-44f9-a2a9-a2b3441f2cbd None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:03.559023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b561a-015b-44f9-a2a9-a2b3441f2cbd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:03.559121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b561a-015b-44f9-a2a9-a2b3441f2cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:03.583659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3a4b561a-015b-44f9-a2a9-a2b3441f2cbd None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:03.585327 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3a4b561a-015b-44f9-a2a9-a2b3441f2cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:03.683189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1223/2442] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:39:03 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:03.701155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c51b04-07c9-4688-9233-47e19c34770a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:03.702627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b7c51b04-07c9-4688-9233-47e19c34770a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:03.815042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c51b04-07c9-4688-9233-47e19c34770a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:03.821251 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a61d2a-f0f6-462b-aeac-88ebec5262ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:03.822069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a61d2a-f0f6-462b-aeac-88ebec5262ca service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:03.822557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a61d2a-f0f6-462b-aeac-88ebec5262ca service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:03.823005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a61d2a-f0f6-462b-aeac-88ebec5262ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:03.834178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a61d2a-f0f6-462b-aeac-88ebec5262ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:03.836038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a61d2a-f0f6-462b-aeac-88ebec5262ca service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:03.846744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1220/2443] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:39:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:39:06.983328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b561a-015b-44f9-a2a9-a2b3441f2cbd None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:07.063031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b561a-015b-44f9-a2a9-a2b3441f2cbd None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:07.067985 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:07.068749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:07.069130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:07.069656 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] PATH_INFO: `/v3/users/5c3d25a4883448d9bedfee863d0d5c66` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:07.070964 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3d25a4883448d9bedfee863d0d5c66)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:07.073030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:07.129192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3d25a4883448d9bedfee863d0d5c66'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:07.155708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3d25a4883448d9bedfee863d0d5c66'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:07.164966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3d25a4883448d9bedfee863d0d5c66'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:07.174933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3d25a4883448d9bedfee863d0d5c66'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:07.175730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3d25a4883448d9bedfee863d0d5c66'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:07.183930 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1224/2444] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:06 2022] DELETE /identity/v3/users/5c3d25a4883448d9bedfee863d0d5c66 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:07.189725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d1379-3a7b-4057-9751-f9814b5ff9f1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:07.190777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d1379-3a7b-4057-9751-f9814b5ff9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:07.191220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d1379-3a7b-4057-9751-f9814b5ff9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:07.219524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-648d1379-3a7b-4057-9751-f9814b5ff9f1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:07.221521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-648d1379-3a7b-4057-9751-f9814b5ff9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:07.332562 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1221/2445] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:39:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:07.341341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:07.343328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:07.396176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:07.466031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e33d57-f19c-4407-a576-d77c0ada2760 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:07.470878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40def187-cf3a-4122-8a28-f35422ef4582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:07.471601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40def187-cf3a-4122-8a28-f35422ef4582 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:07.471928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40def187-cf3a-4122-8a28-f35422ef4582 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:07.472319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40def187-cf3a-4122-8a28-f35422ef4582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:07.488660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40def187-cf3a-4122-8a28-f35422ef4582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:07.490264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40def187-cf3a-4122-8a28-f35422ef4582 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:07.512433 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1225/2446] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:07.940991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d1379-3a7b-4057-9751-f9814b5ff9f1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:07.947172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-648d1379-3a7b-4057-9751-f9814b5ff9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:07.960551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d1379-3a7b-4057-9751-f9814b5ff9f1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:07.980587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:07.981539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:07.981899 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:07.982411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] PATH_INFO: `/v3/projects/a4720f52f41641ed92b785e911a4e0ab` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:07.984685 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4720f52f41641ed92b785e911a4e0ab)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:07.986881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:08.072606 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4720f52f41641ed92b785e911a4e0ab', 'request_id': 'req-8966427b-eb10-4e16-9135-4d202294f1ae'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:08.085477 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4720f52f41641ed92b785e911a4e0ab', 'request_id': 'req-8966427b-eb10-4e16-9135-4d202294f1ae'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:08.094308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4720f52f41641ed92b785e911a4e0ab', 'request_id': 'req-8966427b-eb10-4e16-9135-4d202294f1ae'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:08.117626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] The token cache is being invalidate because project a4720f52f41641ed92b785e911a4e0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:08.118262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:08.119503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8966427b-eb10-4e16-9135-4d202294f1ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:08.121427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1222/2447] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:07 2022] DELETE /identity/v3/projects/a4720f52f41641ed92b785e911a4e0ab => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:08.127653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614acd3f-0368-42c2-afd2-ce286bfb5dae None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:08.128320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614acd3f-0368-42c2-afd2-ce286bfb5dae None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:08.128320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614acd3f-0368-42c2-afd2-ce286bfb5dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:08.154821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-614acd3f-0368-42c2-afd2-ce286bfb5dae None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:08.156723 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-614acd3f-0368-42c2-afd2-ce286bfb5dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:08.231787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1226/2448] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:39:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:08.240073 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab768090-b235-43c3-ac09-c8ad7c564959 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:08.241220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab768090-b235-43c3-ac09-c8ad7c564959 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:08.242501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab768090-b235-43c3-ac09-c8ad7c564959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:08.271621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ab768090-b235-43c3-ac09-c8ad7c564959 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:08.273215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ab768090-b235-43c3-ac09-c8ad7c564959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:08.283025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1223/2449] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:39:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:08.296720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614acd3f-0368-42c2-afd2-ce286bfb5dae None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:08.312797 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614acd3f-0368-42c2-afd2-ce286bfb5dae None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:08.322005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761eeb-04d1-4b4a-88ff-75dc90a0d3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:08.322645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761eeb-04d1-4b4a-88ff-75dc90a0d3af admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:08.322928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761eeb-04d1-4b4a-88ff-75dc90a0d3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:08.323198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761eeb-04d1-4b4a-88ff-75dc90a0d3af admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:08.323886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47761eeb-04d1-4b4a-88ff-75dc90a0d3af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:39:08.324748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47761eeb-04d1-4b4a-88ff-75dc90a0d3af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:08.326214 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47761eeb-04d1-4b4a-88ff-75dc90a0d3af admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:08.341050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1227/2450] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:39:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:39:08.345434 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab768090-b235-43c3-ac09-c8ad7c564959 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:08.356937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab768090-b235-43c3-ac09-c8ad7c564959 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:08.362836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798e585-7c79-4d14-8711-cdf7bd08feee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:08.363571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798e585-7c79-4d14-8711-cdf7bd08feee admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:08.364159 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798e585-7c79-4d14-8711-cdf7bd08feee admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:08.364546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798e585-7c79-4d14-8711-cdf7bd08feee admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:08.365283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798e585-7c79-4d14-8711-cdf7bd08feee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:08.368150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798e585-7c79-4d14-8711-cdf7bd08feee admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:08.373420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:08.408137 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1224/2451] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:08 2022] POST /identity/v3/projects => generated 386 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:08.412233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761eeb-04d1-4b4a-88ff-75dc90a0d3af admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:08.423718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761eeb-04d1-4b4a-88ff-75dc90a0d3af admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:08.491928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aefaf8-35cb-4894-a316-0e36eab69730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:08.492694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aefaf8-35cb-4894-a316-0e36eab69730 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:08.492986 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aefaf8-35cb-4894-a316-0e36eab69730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:08.493254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aefaf8-35cb-4894-a316-0e36eab69730 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:08.493939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aefaf8-35cb-4894-a316-0e36eab69730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:08.495918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aefaf8-35cb-4894-a316-0e36eab69730 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:08.520695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1228/2452] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:08 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:39:08.525547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798e585-7c79-4d14-8711-cdf7bd08feee admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:08.554620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798e585-7c79-4d14-8711-cdf7bd08feee admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:08.566330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f257f86-ae79-47d0-9f1d-16c1dee19a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:08.566959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f257f86-ae79-47d0-9f1d-16c1dee19a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:08.567467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f257f86-ae79-47d0-9f1d-16c1dee19a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:08.567905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f257f86-ae79-47d0-9f1d-16c1dee19a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:08.569284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f257f86-ae79-47d0-9f1d-16c1dee19a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:08.586266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f257f86-ae79-47d0-9f1d-16c1dee19a22 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:08.598983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1225/2453] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:08 2022] GET /identity/v3/roles => generated 1427 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:08.629451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aefaf8-35cb-4894-a316-0e36eab69730 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:08.673117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aefaf8-35cb-4894-a316-0e36eab69730 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:08.684867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfaf86-86d6-4a45-8201-179f4433ba73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:08.685634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfaf86-86d6-4a45-8201-179f4433ba73 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:08.686262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfaf86-86d6-4a45-8201-179f4433ba73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:08.686543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfaf86-86d6-4a45-8201-179f4433ba73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:08.688918 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfaf86-86d6-4a45-8201-179f4433ba73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:08.690629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfaf86-86d6-4a45-8201-179f4433ba73 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:08.721629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1229/2454] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:08 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:08.724874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f257f86-ae79-47d0-9f1d-16c1dee19a22 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:08.743916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f257f86-ae79-47d0-9f1d-16c1dee19a22 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:08.766642 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8108ef-e89e-4cb6-b50c-57e906cc7921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:08.768083 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8108ef-e89e-4cb6-b50c-57e906cc7921 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:08.768083 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8108ef-e89e-4cb6-b50c-57e906cc7921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:08.768083 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8108ef-e89e-4cb6-b50c-57e906cc7921 admin admin] PATH_INFO: `/v3/projects/829a41984faf4b68a3f4b1e2fd2ded27/users/c1bda5eb9abc4f6da200ea3f9500fbfa/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:08.787914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8108ef-e89e-4cb6-b50c-57e906cc7921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=829a41984faf4b68a3f4b1e2fd2ded27, user_id=c1bda5eb9abc4f6da200ea3f9500fbfa, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:08.791725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8108ef-e89e-4cb6-b50c-57e906cc7921 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:08.809097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1226/2455] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:08 2022] PUT /identity/v3/projects/829a41984faf4b68a3f4b1e2fd2ded27/users/c1bda5eb9abc4f6da200ea3f9500fbfa/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:08.814134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d91bb-455e-4e81-ad99-8ce401764de7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:08.814524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d91bb-455e-4e81-ad99-8ce401764de7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:08.814753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d91bb-455e-4e81-ad99-8ce401764de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:08.858622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f97d91bb-455e-4e81-ad99-8ce401764de7 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:08.860069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f97d91bb-455e-4e81-ad99-8ce401764de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:08.956062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:09.018615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1230/2456] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:39:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:09.030886 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8108ef-e89e-4cb6-b50c-57e906cc7921 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:09.035077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ca8108ef-e89e-4cb6-b50c-57e906cc7921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:09.126248 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8108ef-e89e-4cb6-b50c-57e906cc7921 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:09.130287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da972e-e9f6-4a67-9681-ca92a97ee035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:09.134833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da972e-e9f6-4a67-9681-ca92a97ee035 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:09.134833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da972e-e9f6-4a67-9681-ca92a97ee035 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:09.134833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da972e-e9f6-4a67-9681-ca92a97ee035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:09.154330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da972e-e9f6-4a67-9681-ca92a97ee035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:09.158302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da972e-e9f6-4a67-9681-ca92a97ee035 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:09.175060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1227/2457] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:17.025870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5785-2496-4a25-85ed-eb923713ccf3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.026612 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5785-2496-4a25-85ed-eb923713ccf3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.026612 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5785-2496-4a25-85ed-eb923713ccf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.109366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4d2f5785-2496-4a25-85ed-eb923713ccf3 None None] MFA Rules not processed for user `c1bda5eb9abc4f6da200ea3f9500fbfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:17.110789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4d2f5785-2496-4a25-85ed-eb923713ccf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:17.204720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1231/2458] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:39:17 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:17.209074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da972e-e9f6-4a67-9681-ca92a97ee035 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.239534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da972e-e9f6-4a67-9681-ca92a97ee035 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.254905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015145aa-4b59-40fb-b291-7db7115a5ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.258041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015145aa-4b59-40fb-b291-7db7115a5ece admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.259639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015145aa-4b59-40fb-b291-7db7115a5ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.261605 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015145aa-4b59-40fb-b291-7db7115a5ece admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.262541 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015145aa-4b59-40fb-b291-7db7115a5ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.274312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015145aa-4b59-40fb-b291-7db7115a5ece admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.351004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1228/2459] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:39:17 2022] POST /identity/v3/projects => generated 386 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:39:17.353251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f5785-2496-4a25-85ed-eb923713ccf3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.402305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f5785-2496-4a25-85ed-eb923713ccf3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.565709 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b3ce8-75fd-4653-b7ae-b383e344682a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.567000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b3ce8-75fd-4653-b7ae-b383e344682a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.567293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b3ce8-75fd-4653-b7ae-b383e344682a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.567565 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b3ce8-75fd-4653-b7ae-b383e344682a admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.568288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b3ce8-75fd-4653-b7ae-b383e344682a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.574798 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b3ce8-75fd-4653-b7ae-b383e344682a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.614928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1232/2460] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:39:17 2022] POST /identity/v3/users => generated 300 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:39:17.620363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015145aa-4b59-40fb-b291-7db7115a5ece admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.641104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015145aa-4b59-40fb-b291-7db7115a5ece admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.647197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036c607-1afd-4b35-8c4a-265928aa9ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.648228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036c607-1afd-4b35-8c4a-265928aa9ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.648760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036c607-1afd-4b35-8c4a-265928aa9ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.649327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036c607-1afd-4b35-8c4a-265928aa9ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.650374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036c607-1afd-4b35-8c4a-265928aa9ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.652291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036c607-1afd-4b35-8c4a-265928aa9ba6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.670651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1229/2461] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:17 2022] GET /identity/v3/roles => generated 1427 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:17.676638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b3ce8-75fd-4653-b7ae-b383e344682a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.698486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b3ce8-75fd-4653-b7ae-b383e344682a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.709431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f083758-234b-4022-a753-0fc54245f7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.710594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f083758-234b-4022-a753-0fc54245f7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.711148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f083758-234b-4022-a753-0fc54245f7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.711715 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f083758-234b-4022-a753-0fc54245f7e5 admin admin] PATH_INFO: `/v3/projects/6fb0902803ac42eda4b0e3f9292f9906/users/1d45cdd023884945adc6783ba94fac79/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.738072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f083758-234b-4022-a753-0fc54245f7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb0902803ac42eda4b0e3f9292f9906, user_id=1d45cdd023884945adc6783ba94fac79, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.742731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f083758-234b-4022-a753-0fc54245f7e5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.796222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1233/2462] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:17 2022] PUT /identity/v3/projects/6fb0902803ac42eda4b0e3f9292f9906/users/1d45cdd023884945adc6783ba94fac79/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:17.799591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036c607-1afd-4b35-8c4a-265928aa9ba6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:17.818620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036c607-1afd-4b35-8c4a-265928aa9ba6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:17.974123 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dd713-5161-4c72-a690-71ad76e4ac4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:17.974862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dd713-5161-4c72-a690-71ad76e4ac4e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:17.975143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dd713-5161-4c72-a690-71ad76e4ac4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:17.976246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dd713-5161-4c72-a690-71ad76e4ac4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:17.976670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982dd713-5161-4c72-a690-71ad76e4ac4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:17.978670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982dd713-5161-4c72-a690-71ad76e4ac4e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:17.999379 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1230/2463] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:39:17 2022] GET /identity/v3/roles => generated 1427 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:39:18.003278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f083758-234b-4022-a753-0fc54245f7e5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:18.022393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f083758-234b-4022-a753-0fc54245f7e5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:18.029597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a94bfa-d1fa-4d9b-8ab5-364e96e7da57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:18.050638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a94bfa-d1fa-4d9b-8ab5-364e96e7da57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:18.051957 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a94bfa-d1fa-4d9b-8ab5-364e96e7da57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:18.052494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a94bfa-d1fa-4d9b-8ab5-364e96e7da57 admin admin] PATH_INFO: `/v3/projects/6fb0902803ac42eda4b0e3f9292f9906/users/1d45cdd023884945adc6783ba94fac79/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:18.059253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a94bfa-d1fa-4d9b-8ab5-364e96e7da57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb0902803ac42eda4b0e3f9292f9906, user_id=1d45cdd023884945adc6783ba94fac79, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:18.062850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a94bfa-d1fa-4d9b-8ab5-364e96e7da57 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:18.113767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1234/2464] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:39:18 2022] PUT /identity/v3/projects/6fb0902803ac42eda4b0e3f9292f9906/users/1d45cdd023884945adc6783ba94fac79/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:24.886762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2bc99-1cd7-4993-8d72-8fff67def64c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:24.888385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2bc99-1cd7-4993-8d72-8fff67def64c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:24.889165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2bc99-1cd7-4993-8d72-8fff67def64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:24.955004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cfc2bc99-1cd7-4993-8d72-8fff67def64c None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:24.960921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cfc2bc99-1cd7-4993-8d72-8fff67def64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:24.989920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:25.088507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1231/2465] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:39:24 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:25.102937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a94bfa-d1fa-4d9b-8ab5-364e96e7da57 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:25.105611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-01a94bfa-d1fa-4d9b-8ab5-364e96e7da57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:25.208351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:25.502157 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a94bfa-d1fa-4d9b-8ab5-364e96e7da57 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:25.525926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dd210-988c-46ec-afdb-1882cc4d94bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:25.526707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dd210-988c-46ec-afdb-1882cc4d94bb service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:25.527088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dd210-988c-46ec-afdb-1882cc4d94bb service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:25.527365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dd210-988c-46ec-afdb-1882cc4d94bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:25.539305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193dd210-988c-46ec-afdb-1882cc4d94bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:25.541356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193dd210-988c-46ec-afdb-1882cc4d94bb service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:25.552185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1235/2466] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:39:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:39:27.251875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b20f8-2cf8-46ac-a77e-7845458fd96b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:27.254233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b20f8-2cf8-46ac-a77e-7845458fd96b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:27.258152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b20f8-2cf8-46ac-a77e-7845458fd96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:27.339024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-970b20f8-2cf8-46ac-a77e-7845458fd96b None None] MFA Rules not processed for user `1d45cdd023884945adc6783ba94fac79`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:27.340566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-970b20f8-2cf8-46ac-a77e-7845458fd96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:27.536237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1232/2467] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:39:27 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:34.574851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dd210-988c-46ec-afdb-1882cc4d94bb service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:34.576294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-193dd210-988c-46ec-afdb-1882cc4d94bb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:34.752737 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dd210-988c-46ec-afdb-1882cc4d94bb service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:34.758037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51072950-96ec-4b37-903e-991696f2317a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0891d81e6e24ad99a9b532059c8c2af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:34.758912 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51072950-96ec-4b37-903e-991696f2317a service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:34.759349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51072950-96ec-4b37-903e-991696f2317a service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:34.759849 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51072950-96ec-4b37-903e-991696f2317a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:34.863219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51072950-96ec-4b37-903e-991696f2317a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:34.864681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51072950-96ec-4b37-903e-991696f2317a service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:34.877906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1236/2468] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:34 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:35.145359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5e8d-2ce3-4887-b8c0-8d26944e40bb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:35.145359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5e8d-2ce3-4887-b8c0-8d26944e40bb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:35.145359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5e8d-2ce3-4887-b8c0-8d26944e40bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:35.162072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-472d5e8d-2ce3-4887-b8c0-8d26944e40bb None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:35.164688 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-472d5e8d-2ce3-4887-b8c0-8d26944e40bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:35.174761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1233/2469] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:39:35 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:35.214917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51072950-96ec-4b37-903e-991696f2317a service glance] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:35.228747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51072950-96ec-4b37-903e-991696f2317a service glance] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:35.235373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11073076-0f8a-481d-9cb5-7296d63f637b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:35.236020 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11073076-0f8a-481d-9cb5-7296d63f637b service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:35.236300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11073076-0f8a-481d-9cb5-7296d63f637b service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:35.237430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11073076-0f8a-481d-9cb5-7296d63f637b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:35.251695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11073076-0f8a-481d-9cb5-7296d63f637b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:35.257935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11073076-0f8a-481d-9cb5-7296d63f637b service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:35.282953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1237/2470] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:39:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:39:40.455912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec444af2-a9f3-4416-bb5d-dc4d63ae6b0d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:40.456470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec444af2-a9f3-4416-bb5d-dc4d63ae6b0d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:40.456470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec444af2-a9f3-4416-bb5d-dc4d63ae6b0d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:40.459495 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1234/2471] 10.222.0.100 () {54 vars in 966 bytes} [Thu Sep 1 17:39:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:39:40.476815 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaaa61-2176-40ee-bf62-19b2ee94b4d6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:40.477891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaaa61-2176-40ee-bf62-19b2ee94b4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:40.478485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaaa61-2176-40ee-bf62-19b2ee94b4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:40.536089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-72eaaa61-2176-40ee-bf62-19b2ee94b4d6 None None] MFA Rules not processed for user `4b181d38588d44d4bb72fa319d2ba2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:40.538610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-72eaaa61-2176-40ee-bf62-19b2ee94b4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:40.599074 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:40.630561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1238/2472] 10.222.0.100 () {58 vars in 1063 bytes} [Thu Sep 1 17:39:40 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:40.637941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec444af2-a9f3-4416-bb5d-dc4d63ae6b0d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:40.681618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec444af2-a9f3-4416-bb5d-dc4d63ae6b0d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:40.681618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1ef8-0124-4614-843c-3387d000778b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:40.682344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1ef8-0124-4614-843c-3387d000778b service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:40.682721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1ef8-0124-4614-843c-3387d000778b service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:40.683095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1ef8-0124-4614-843c-3387d000778b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:40.684957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-309b1ef8-0124-4614-843c-3387d000778b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:40.702945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b1ef8-0124-4614-843c-3387d000778b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:40.704965 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b1ef8-0124-4614-843c-3387d000778b service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:40.737271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1235/2473] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:40 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:41.001703 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaaa61-2176-40ee-bf62-19b2ee94b4d6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:41.166821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaaa61-2176-40ee-bf62-19b2ee94b4d6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:41.168041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094a437-4802-46c7-8257-fecc0ca60ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:41.168764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094a437-4802-46c7-8257-fecc0ca60ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:41.169154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094a437-4802-46c7-8257-fecc0ca60ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:41.169491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094a437-4802-46c7-8257-fecc0ca60ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:41.171457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a094a437-4802-46c7-8257-fecc0ca60ec6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:41.368186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094a437-4802-46c7-8257-fecc0ca60ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:41.369799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094a437-4802-46c7-8257-fecc0ca60ec6 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:41.380942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1239/2474] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:39:42.544114 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a28397-5bdd-4bd9-a98e-2f1330e49543 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:42.545068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a28397-5bdd-4bd9-a98e-2f1330e49543 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:42.545511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a28397-5bdd-4bd9-a98e-2f1330e49543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:42.567973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d7a28397-5bdd-4bd9-a98e-2f1330e49543 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:42.570064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d7a28397-5bdd-4bd9-a98e-2f1330e49543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:42.585674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1236/2475] 10.222.0.100 () {58 vars in 972 bytes} [Thu Sep 1 17:39:42 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:42.604911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094a437-4802-46c7-8257-fecc0ca60ec6 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:42.624567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094a437-4802-46c7-8257-fecc0ca60ec6 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:42.633924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdb39a-2b6f-4a21-80b0-bd66e8ffbdf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:42.635656 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdb39a-2b6f-4a21-80b0-bd66e8ffbdf7 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:42.636117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdb39a-2b6f-4a21-80b0-bd66e8ffbdf7 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:42.636561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdb39a-2b6f-4a21-80b0-bd66e8ffbdf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:42.647078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdb39a-2b6f-4a21-80b0-bd66e8ffbdf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:42.658118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdb39a-2b6f-4a21-80b0-bd66e8ffbdf7 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:42.666632 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1240/2476] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:39:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:39:45.561478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a28397-5bdd-4bd9-a98e-2f1330e49543 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:45.563621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d7a28397-5bdd-4bd9-a98e-2f1330e49543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:45.697498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a28397-5bdd-4bd9-a98e-2f1330e49543 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:45.705214 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:45.706507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:45.707339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:45.707954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] PATH_INFO: `/v3/users/75de510214004bbeab187d65bf178735` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:45.710439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] RBAC: Authorizing `identity:delete_user(user_id=75de510214004bbeab187d65bf178735)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:45.713476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:45.750100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:45.794731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75de510214004bbeab187d65bf178735'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:45.808227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75de510214004bbeab187d65bf178735'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:45.809414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75de510214004bbeab187d65bf178735'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:45.843613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75de510214004bbeab187d65bf178735'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:45.854889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a9ff4a14-f1f0-48e2-b33a-8fc48e4bd52d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75de510214004bbeab187d65bf178735'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:45.862589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1237/2477] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:39:45 2022] DELETE /identity/v3/users/75de510214004bbeab187d65bf178735 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:46.196602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdb39a-2b6f-4a21-80b0-bd66e8ffbdf7 service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:46.248271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdb39a-2b6f-4a21-80b0-bd66e8ffbdf7 service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:46.453224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:46.453224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:46.453224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:46.453224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] PATH_INFO: `/v3/projects/356d731e34b24c9d8852516ae9141124` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:46.462324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=356d731e34b24c9d8852516ae9141124)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:46.467078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:46.534935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:46.608797 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '356d731e34b24c9d8852516ae9141124', 'request_id': 'req-634d55a9-4579-4e5c-a030-4d4815fca5b5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:46.633992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '356d731e34b24c9d8852516ae9141124', 'request_id': 'req-634d55a9-4579-4e5c-a030-4d4815fca5b5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:46.642501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '356d731e34b24c9d8852516ae9141124', 'request_id': 'req-634d55a9-4579-4e5c-a030-4d4815fca5b5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:46.672265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] The token cache is being invalidate because project 356d731e34b24c9d8852516ae9141124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:39:46.672643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:46.673946 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-634d55a9-4579-4e5c-a030-4d4815fca5b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:39:46.690510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1241/2478] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:39:46 2022] DELETE /identity/v3/projects/356d731e34b24c9d8852516ae9141124 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:39:48.298266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec2c55-ccdd-46b2-abb2-fa1df50bbce5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:48.299976 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec2c55-ccdd-46b2-abb2-fa1df50bbce5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:48.300427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec2c55-ccdd-46b2-abb2-fa1df50bbce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:48.339627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-82ec2c55-ccdd-46b2-abb2-fa1df50bbce5 None None] MFA Rules not processed for user `c1bda5eb9abc4f6da200ea3f9500fbfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:48.341348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-82ec2c55-ccdd-46b2-abb2-fa1df50bbce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:48.424491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1238/2479] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:39:48 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:39:49.015202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aebc7-2a13-49a4-9a68-38af9c21b595 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.015786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aebc7-2a13-49a4-9a68-38af9c21b595 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.016119 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aebc7-2a13-49a4-9a68-38af9c21b595 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.017631 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1242/2480] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:39:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:39:49.025874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417a111-a041-440b-8945-ae4ab61202a8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.026157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417a111-a041-440b-8945-ae4ab61202a8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.026436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417a111-a041-440b-8945-ae4ab61202a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.049998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d417a111-a041-440b-8945-ae4ab61202a8 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:49.051359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d417a111-a041-440b-8945-ae4ab61202a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:49.145463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1239/2481] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:39:49 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:49.163142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4aebc7-2a13-49a4-9a68-38af9c21b595 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.164895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cc4aebc7-2a13-49a4-9a68-38af9c21b595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:49.238446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4aebc7-2a13-49a4-9a68-38af9c21b595 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.243550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcb8a7-3290-40c5-9e8f-723eedeec4ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0891d81e6e24ad99a9b532059c8c2af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.244178 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcb8a7-3290-40c5-9e8f-723eedeec4ff service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.245028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcb8a7-3290-40c5-9e8f-723eedeec4ff service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.245028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcb8a7-3290-40c5-9e8f-723eedeec4ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.257810 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcb8a7-3290-40c5-9e8f-723eedeec4ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.259268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcb8a7-3290-40c5-9e8f-723eedeec4ff service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.271136 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1243/2482] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:49 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:49.348489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab213c-ce93-4a99-86b6-316064ddadd1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.349014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab213c-ce93-4a99-86b6-316064ddadd1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.349131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab213c-ce93-4a99-86b6-316064ddadd1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.350620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1240/2483] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:39:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:39:49.357854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327fdc3-d5ea-4dc0-b487-d467d82a015d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.358146 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327fdc3-d5ea-4dc0-b487-d467d82a015d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.358419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327fdc3-d5ea-4dc0-b487-d467d82a015d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.379280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5327fdc3-d5ea-4dc0-b487-d467d82a015d None None] MFA Rules not processed for user `bbe180b45f2e49e2abbf6e833ce0d44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:49.380640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5327fdc3-d5ea-4dc0-b487-d467d82a015d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:49.446389 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1244/2484] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:39:49 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:49.458468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab213c-ce93-4a99-86b6-316064ddadd1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.462155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-94ab213c-ce93-4a99-86b6-316064ddadd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:49.480777 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab213c-ce93-4a99-86b6-316064ddadd1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.484526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6e2ed-5c4b-4ee7-8f11-3e34167b3544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.485975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6e2ed-5c4b-4ee7-8f11-3e34167b3544 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.485975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6e2ed-5c4b-4ee7-8f11-3e34167b3544 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.485975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6e2ed-5c4b-4ee7-8f11-3e34167b3544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.494941 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6e2ed-5c4b-4ee7-8f11-3e34167b3544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.496487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6e2ed-5c4b-4ee7-8f11-3e34167b3544 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.506470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1241/2485] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:49 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:49.604335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255d8e-ad3a-46c9-b570-b6cffbd9fa33 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.604624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255d8e-ad3a-46c9-b570-b6cffbd9fa33 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.605030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255d8e-ad3a-46c9-b570-b6cffbd9fa33 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.606696 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1245/2486] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:39:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:39:49.611549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009e1cd-3d73-4013-8cc3-2a925a734a77 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.611839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009e1cd-3d73-4013-8cc3-2a925a734a77 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.612116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009e1cd-3d73-4013-8cc3-2a925a734a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.634044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e009e1cd-3d73-4013-8cc3-2a925a734a77 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:49.635310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e009e1cd-3d73-4013-8cc3-2a925a734a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:49.645347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1242/2487] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:39:49 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:49.658425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255d8e-ad3a-46c9-b570-b6cffbd9fa33 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:49.660595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c8255d8e-ad3a-46c9-b570-b6cffbd9fa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:49.677188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255d8e-ad3a-46c9-b570-b6cffbd9fa33 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:49.682406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e7239-d79b-4b1c-bbc1-fdc8e1d3c284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:49.683939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e7239-d79b-4b1c-bbc1-fdc8e1d3c284 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:49.683939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e7239-d79b-4b1c-bbc1-fdc8e1d3c284 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:49.683939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e7239-d79b-4b1c-bbc1-fdc8e1d3c284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:49.703635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e7239-d79b-4b1c-bbc1-fdc8e1d3c284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:49.705105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e7239-d79b-4b1c-bbc1-fdc8e1d3c284 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:49.718610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1246/2488] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:39:49 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:39:51.027613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45075ffa-0866-4d1e-a55c-df3f3931f025 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.028124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45075ffa-0866-4d1e-a55c-df3f3931f025 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.028124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45075ffa-0866-4d1e-a55c-df3f3931f025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.049607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-45075ffa-0866-4d1e-a55c-df3f3931f025 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:39:51.050847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-45075ffa-0866-4d1e-a55c-df3f3931f025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:51.066901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:51.126812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1243/2489] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:39:51 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:39:51.142115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e7239-d79b-4b1c-bbc1-fdc8e1d3c284 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:39:51.143569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-638e7239-d79b-4b1c-bbc1-fdc8e1d3c284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:39:51.217839 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:39:51.249435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e7239-d79b-4b1c-bbc1-fdc8e1d3c284 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:39:51.255633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9068804-bfa3-4c4b-9915-e627621c817b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:39:51.256399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9068804-bfa3-4c4b-9915-e627621c817b service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:39:51.256847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9068804-bfa3-4c4b-9915-e627621c817b service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:39:51.257315 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9068804-bfa3-4c4b-9915-e627621c817b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:39:51.269711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9068804-bfa3-4c4b-9915-e627621c817b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:39:51.272340 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9068804-bfa3-4c4b-9915-e627621c817b service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:39:51.293530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1247/2490] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:39:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:40:01.119982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=18 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:01.279704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=10 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:13.017227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a5c59-b200-4f7b-8351-174230d0c153 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:13.017754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a5c59-b200-4f7b-8351-174230d0c153 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:13.017850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a5c59-b200-4f7b-8351-174230d0c153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:13.039201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fd0a5c59-b200-4f7b-8351-174230d0c153 None None] MFA Rules not processed for user `d3848643d2c44951b78814ccf7d504af`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:13.040386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fd0a5c59-b200-4f7b-8351-174230d0c153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:13.095971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1244/2491] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:40:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:18.211711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f18c74-d6aa-43fb-93c3-840657f309d4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:18.212209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f18c74-d6aa-43fb-93c3-840657f309d4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:18.212336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f18c74-d6aa-43fb-93c3-840657f309d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:18.246418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-88f18c74-d6aa-43fb-93c3-840657f309d4 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:18.248823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-88f18c74-d6aa-43fb-93c3-840657f309d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:18.262461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1248/2492] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:40:18 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:18.278165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a5c59-b200-4f7b-8351-174230d0c153 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:18.301189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a5c59-b200-4f7b-8351-174230d0c153 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:18.319879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79c74d-7e65-4f61-89ce-988380c9d6e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:18.320935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79c74d-7e65-4f61-89ce-988380c9d6e5 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:18.321473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79c74d-7e65-4f61-89ce-988380c9d6e5 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:18.321991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79c74d-7e65-4f61-89ce-988380c9d6e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:18.339973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79c74d-7e65-4f61-89ce-988380c9d6e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:18.342053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79c74d-7e65-4f61-89ce-988380c9d6e5 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:18.354007 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1245/2493] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:40:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:40:28.234204 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:28.350010 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=31 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:40:40.213072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f18c74-d6aa-43fb-93c3-840657f309d4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:40.224259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f18c74-d6aa-43fb-93c3-840657f309d4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:40.229230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1234d8-4613-4e4a-b4a3-a29b3d5d4bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:40.229897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1234d8-4613-4e4a-b4a3-a29b3d5d4bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:40.230177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1234d8-4613-4e4a-b4a3-a29b3d5d4bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:40.230448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1234d8-4613-4e4a-b4a3-a29b3d5d4bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:40.232435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ba1234d8-4613-4e4a-b4a3-a29b3d5d4bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:40.302570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1234d8-4613-4e4a-b4a3-a29b3d5d4bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:40.304017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1234d8-4613-4e4a-b4a3-a29b3d5d4bb4 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:40.314041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1249/2494] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:40:40 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:40:45.137771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd283b-483a-4593-bb3f-4cbbc9b4c06a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:45.138813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd283b-483a-4593-bb3f-4cbbc9b4c06a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:45.139097 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd283b-483a-4593-bb3f-4cbbc9b4c06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:45.159732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-26bd283b-483a-4593-bb3f-4cbbc9b4c06a None None] MFA Rules not processed for user `66366eb7b7044786b23d08676c14670f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:45.161803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-26bd283b-483a-4593-bb3f-4cbbc9b4c06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:45.243812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1246/2495] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:40:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:47.875413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b59e93-1647-4eb2-ad5f-a6e936b0abf8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:47.876730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b59e93-1647-4eb2-ad5f-a6e936b0abf8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:47.877158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b59e93-1647-4eb2-ad5f-a6e936b0abf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:47.919502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-42b59e93-1647-4eb2-ad5f-a6e936b0abf8 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:47.928143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-42b59e93-1647-4eb2-ad5f-a6e936b0abf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:47.964473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1250/2496] 10.222.0.100 () {58 vars in 972 bytes} [Thu Sep 1 17:40:47 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:40:48.000970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd283b-483a-4593-bb3f-4cbbc9b4c06a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:40:48.031255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd283b-483a-4593-bb3f-4cbbc9b4c06a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:40:48.040584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c95e6-0b8f-4933-848c-cb838a9baa56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:40:48.041218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c95e6-0b8f-4933-848c-cb838a9baa56 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:48.041726 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c95e6-0b8f-4933-848c-cb838a9baa56 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:48.042186 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c95e6-0b8f-4933-848c-cb838a9baa56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:48.057866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c95e6-0b8f-4933-848c-cb838a9baa56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:40:48.060806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c95e6-0b8f-4933-848c-cb838a9baa56 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:40:48.078684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1247/2497] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:40:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:40:53.700813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f16f19-f6db-4a56-b166-538201117300 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:40:53.701534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f16f19-f6db-4a56-b166-538201117300 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:40:53.701534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f16f19-f6db-4a56-b166-538201117300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:40:53.721866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-34f16f19-f6db-4a56-b166-538201117300 None None] MFA Rules not processed for user `5647214c5b3f438eb3ef1ddc14153d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:40:53.723914 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34f16f19-f6db-4a56-b166-538201117300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:40:53.812869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1251/2498] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:40:53 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:40:58.071290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=31 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:03.800543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=58 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:28.417022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c95e6-0b8f-4933-848c-cb838a9baa56 service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:28.426567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c95e6-0b8f-4933-848c-cb838a9baa56 service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:28.431927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dc106-d151-4fd5-84e4-f87f272224e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:28.432521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dc106-d151-4fd5-84e4-f87f272224e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:28.432795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dc106-d151-4fd5-84e4-f87f272224e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:28.433089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dc106-d151-4fd5-84e4-f87f272224e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:28.434816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ae1dc106-d151-4fd5-84e4-f87f272224e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:28.449465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1dc106-d151-4fd5-84e4-f87f272224e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:28.452574 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1dc106-d151-4fd5-84e4-f87f272224e3 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:28.465382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1248/2499] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:28 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:38.460282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:43.204427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f16f19-f6db-4a56-b166-538201117300 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:43.214845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f16f19-f6db-4a56-b166-538201117300 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:43.218472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:43.219085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:43.219368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:43.219640 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:43.221388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:43.235942 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:43.237379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:43.249254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1252/2500] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:43 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:46.267044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dc106-d151-4fd5-84e4-f87f272224e3 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:46.276492 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dc106-d151-4fd5-84e4-f87f272224e3 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:46.280979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00427f9-5f8d-410e-b140-d7edaa475e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:46.281628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00427f9-5f8d-410e-b140-d7edaa475e36 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:46.281922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00427f9-5f8d-410e-b140-d7edaa475e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:46.282192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00427f9-5f8d-410e-b140-d7edaa475e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:46.283970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c00427f9-5f8d-410e-b140-d7edaa475e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:46.359678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00427f9-5f8d-410e-b140-d7edaa475e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:46.361521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00427f9-5f8d-410e-b140-d7edaa475e36 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:46.375156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1249/2501] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:50.046811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:50.048449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:50.175783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380affa-c573-41c5-b297-d432c4833006 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:50.180955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf094cc-65a6-4b08-9390-695648326b35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:50.181833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf094cc-65a6-4b08-9390-695648326b35 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:50.182186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf094cc-65a6-4b08-9390-695648326b35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:50.182755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf094cc-65a6-4b08-9390-695648326b35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:50.201417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-acf094cc-65a6-4b08-9390-695648326b35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:50.225510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf094cc-65a6-4b08-9390-695648326b35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:50.227243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf094cc-65a6-4b08-9390-695648326b35 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:50.245344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1253/2502] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:50 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:52.315653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc03f34-c2ac-4768-8f1b-43d4b93937b4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:52.317075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc03f34-c2ac-4768-8f1b-43d4b93937b4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:52.317546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc03f34-c2ac-4768-8f1b-43d4b93937b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:52.345796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3bc03f34-c2ac-4768-8f1b-43d4b93937b4 None None] MFA Rules not processed for user `bcc538bc852f404699d91d17023ae620`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:41:52.348009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3bc03f34-c2ac-4768-8f1b-43d4b93937b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:52.356996 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1250/2503] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:41:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:41:52.719636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf094cc-65a6-4b08-9390-695648326b35 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:52.730264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf094cc-65a6-4b08-9390-695648326b35 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:52.733932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:52.734543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:52.734820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:52.735090 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:52.736919 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:52.748055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:52.749489 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:52.759055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1254/2504] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:52 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:53.128949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc03f34-c2ac-4768-8f1b-43d4b93937b4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:53.130555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3bc03f34-c2ac-4768-8f1b-43d4b93937b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:53.223400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc03f34-c2ac-4768-8f1b-43d4b93937b4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:53.233382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e253c-65e1-419b-aa04-d0e94161952b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:53.234050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e253c-65e1-419b-aa04-d0e94161952b service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:53.234334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e253c-65e1-419b-aa04-d0e94161952b service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:53.234733 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e253c-65e1-419b-aa04-d0e94161952b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:53.236741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f32e253c-65e1-419b-aa04-d0e94161952b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:53.253817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e253c-65e1-419b-aa04-d0e94161952b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:53.256174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e253c-65e1-419b-aa04-d0e94161952b service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:53.274689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1251/2505] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:53 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:41:54.157285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:54.158757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:54.231617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf185b-f86e-4623-af59-858e0afd7422 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:54.237636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:54.237874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:54.238025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:54.238358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] PATH_INFO: `/v3/users/66366eb7b7044786b23d08676c14670f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:54.240084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66366eb7b7044786b23d08676c14670f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:54.242467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:54.319669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66366eb7b7044786b23d08676c14670f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.334691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66366eb7b7044786b23d08676c14670f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.343522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66366eb7b7044786b23d08676c14670f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.358269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:54.358269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66366eb7b7044786b23d08676c14670f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.358269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66366eb7b7044786b23d08676c14670f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.364298 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1255/2506] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:41:54 2022] DELETE /identity/v3/users/66366eb7b7044786b23d08676c14670f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:41:54.615931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e253c-65e1-419b-aa04-d0e94161952b service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:54.630324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e253c-65e1-419b-aa04-d0e94161952b service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:54.696988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:54.697676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:54.697957 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:41:54.698321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] PATH_INFO: `/v3/projects/6695ac82241740f5ac5f73b4bcdac5f5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:41:54.699599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6695ac82241740f5ac5f73b4bcdac5f5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:41:54.701625 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:41:54.728890 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:41:54.754147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6695ac82241740f5ac5f73b4bcdac5f5', 'request_id': 'req-de76a40b-e6fc-434d-846a-20091349ee3e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.761694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6695ac82241740f5ac5f73b4bcdac5f5', 'request_id': 'req-de76a40b-e6fc-434d-846a-20091349ee3e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.768294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6695ac82241740f5ac5f73b4bcdac5f5', 'request_id': 'req-de76a40b-e6fc-434d-846a-20091349ee3e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.783076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] The token cache is being invalidate because project 6695ac82241740f5ac5f73b4bcdac5f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:41:54.783302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.784203 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:41:54.785861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1252/2507] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:41:54 2022] DELETE /identity/v3/projects/6695ac82241740f5ac5f73b4bcdac5f5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:41:59.838325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:41:59.839950 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:41:59.941219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab87393-ffe7-40ad-a332-6cf65ad41961 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:41:59.998126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:41:59.999520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:41:59.999888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:00.000436 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:00.002914 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:00.016809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:00.018880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:00.031655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1256/2508] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:41:59 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:00.288405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:00.290334 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:00.541004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76a40b-e6fc-434d-846a-20091349ee3e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:00.563706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c4596-7e1e-4ed4-a57c-373619c8600a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:00.566132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c4596-7e1e-4ed4-a57c-373619c8600a service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:00.566427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c4596-7e1e-4ed4-a57c-373619c8600a service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:00.566741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c4596-7e1e-4ed4-a57c-373619c8600a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:00.575871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-525c4596-7e1e-4ed4-a57c-373619c8600a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:00.635352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c4596-7e1e-4ed4-a57c-373619c8600a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:00.661757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c4596-7e1e-4ed4-a57c-373619c8600a service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:00.689309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1253/2509] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:00 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:01.150098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:01.152142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:01.182359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342977df-a74a-4575-944b-1fce697a0e0f service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:01.192323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:01.194440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:01.195300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:01.195792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:01.198347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:01.216066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:01.219835 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:01.238300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1257/2510] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:01 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:04.451883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f93714-8b74-42cd-b16e-cb441bbdd1f6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:04.452417 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f93714-8b74-42cd-b16e-cb441bbdd1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:04.452861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f93714-8b74-42cd-b16e-cb441bbdd1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:04.476592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b6f93714-8b74-42cd-b16e-cb441bbdd1f6 None None] MFA Rules not processed for user `d3848643d2c44951b78814ccf7d504af`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:04.477982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b6f93714-8b74-42cd-b16e-cb441bbdd1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:04.537999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1254/2511] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:42:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:42:04.864477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:04.866097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:04.877649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d266f-aa12-4d01-8cef-145d4bb5f37c service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:04.881367 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7a00a-53da-4fe4-af16-95d411a647a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:04.881995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7a00a-53da-4fe4-af16-95d411a647a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:04.882277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7a00a-53da-4fe4-af16-95d411a647a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:04.882553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7a00a-53da-4fe4-af16-95d411a647a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:04.884266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b1a7a00a-53da-4fe4-af16-95d411a647a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:04.893986 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7a00a-53da-4fe4-af16-95d411a647a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:04.895404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7a00a-53da-4fe4-af16-95d411a647a0 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:04.904293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1258/2512] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:04 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:14.530598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=74,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:14.900788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=60 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:21.730169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f93714-8b74-42cd-b16e-cb441bbdd1f6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:21.740199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f93714-8b74-42cd-b16e-cb441bbdd1f6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:21.743819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:21.744427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:21.744708 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:21.744981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:21.746741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:21.814012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:21.815545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:21.827199 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1255/2513] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:23.400747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcf0ea-061b-484e-ab10-2617a8231d4d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:23.404885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcf0ea-061b-484e-ab10-2617a8231d4d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:23.404885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcf0ea-061b-484e-ab10-2617a8231d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:23.428917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-38dcf0ea-061b-484e-ab10-2617a8231d4d None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:23.430414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-38dcf0ea-061b-484e-ab10-2617a8231d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:23.500732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1259/2514] 10.222.0.100 () {58 vars in 972 bytes} [Thu Sep 1 17:42:23 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:42:23.557131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:23.558654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:23.671806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c9301-f7a7-4755-b4f2-57544a3e3ccf service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:23.677961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b09ae-09e8-45e9-814b-aeff56db48b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:23.678713 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b09ae-09e8-45e9-814b-aeff56db48b6 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:23.678910 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b09ae-09e8-45e9-814b-aeff56db48b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:23.679184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b09ae-09e8-45e9-814b-aeff56db48b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:23.688996 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b09ae-09e8-45e9-814b-aeff56db48b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:23.690694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b09ae-09e8-45e9-814b-aeff56db48b6 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:23.700949 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1256/2515] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:42:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:42:24.626080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ca1ca-efa3-4b15-a609-575da98ca50b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:24.627102 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ca1ca-efa3-4b15-a609-575da98ca50b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:24.627590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ca1ca-efa3-4b15-a609-575da98ca50b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:24.629520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1260/2516] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:42:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:42:24.637264 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a598d-498e-4a86-90fb-981d5c75d54f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:24.638235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a598d-498e-4a86-90fb-981d5c75d54f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:24.638724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a598d-498e-4a86-90fb-981d5c75d54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:24.670740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b01a598d-498e-4a86-90fb-981d5c75d54f None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:24.671996 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b01a598d-498e-4a86-90fb-981d5c75d54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:24.686246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1257/2517] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:42:24 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:42:24.720449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ca1ca-efa3-4b15-a609-575da98ca50b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:24.722586 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b88ca1ca-efa3-4b15-a609-575da98ca50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:24.742475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ca1ca-efa3-4b15-a609-575da98ca50b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:24.747713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b798a8-3f9a-4964-b885-125be0666bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:24.748644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b798a8-3f9a-4964-b885-125be0666bfd service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:24.748927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b798a8-3f9a-4964-b885-125be0666bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:24.750077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b798a8-3f9a-4964-b885-125be0666bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:24.771173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b798a8-3f9a-4964-b885-125be0666bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:24.772897 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b798a8-3f9a-4964-b885-125be0666bfd service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:24.786557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1261/2518] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:24 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:26.849002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b807a-2d8d-410b-9301-346a853f99fc None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:26.850510 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b807a-2d8d-410b-9301-346a853f99fc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:26.850993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b807a-2d8d-410b-9301-346a853f99fc None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:26.853262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1258/2519] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:42:26 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Sep 01 17:42:26.883067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41537228-3439-494f-9e2a-e097146eca1b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:26.883577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41537228-3439-494f-9e2a-e097146eca1b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:26.883861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41537228-3439-494f-9e2a-e097146eca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:26.908632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b807a-2d8d-410b-9301-346a853f99fc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:26.951797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b807a-2d8d-410b-9301-346a853f99fc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:26.968347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-41537228-3439-494f-9e2a-e097146eca1b None None] MFA Rules not processed for user `4b181d38588d44d4bb72fa319d2ba2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:26.971015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-41537228-3439-494f-9e2a-e097146eca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:26.971819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:26.974042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:26.974753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:26.975939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:26.981330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:27.001253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:27.003425 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:27.024936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1259/2520] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:26 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:27.076419 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:27.078431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:27.113920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd9d1-f14d-4eb4-a9aa-75987ae29470 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:27.119620 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1262/2521] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:42:26 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:42:27.124291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:27.126072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:27.127166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:27.127621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] PATH_INFO: `/v3/users/5647214c5b3f438eb3ef1ddc14153d9e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:27.129116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5647214c5b3f438eb3ef1ddc14153d9e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:27.131666 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:27.142562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41537228-3439-494f-9e2a-e097146eca1b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:27.146421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-41537228-3439-494f-9e2a-e097146eca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:27.168599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41537228-3439-494f-9e2a-e097146eca1b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:27.187560 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea73a9e-23fc-4f61-b87f-03c837425085 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:27.192661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea73a9e-23fc-4f61-b87f-03c837425085 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:27.193193 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea73a9e-23fc-4f61-b87f-03c837425085 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:27.193671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea73a9e-23fc-4f61-b87f-03c837425085 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:27.220152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea73a9e-23fc-4f61-b87f-03c837425085 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:27.229670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea73a9e-23fc-4f61-b87f-03c837425085 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:27.248418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5647214c5b3f438eb3ef1ddc14153d9e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.258744 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5647214c5b3f438eb3ef1ddc14153d9e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.260051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1263/2522] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:27 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:42:27.260294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5647214c5b3f438eb3ef1ddc14153d9e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.266449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:27.278765 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5647214c5b3f438eb3ef1ddc14153d9e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.290065 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4894dcb0-e717-4024-bf0d-d5f65376e9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5647214c5b3f438eb3ef1ddc14153d9e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.298059 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1260/2523] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:42:27 2022] DELETE /identity/v3/users/5647214c5b3f438eb3ef1ddc14153d9e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:42:27.737013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea73a9e-23fc-4f61-b87f-03c837425085 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:27.764537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea73a9e-23fc-4f61-b87f-03c837425085 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:27.880092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:27.881232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:27.882032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:27.882642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] PATH_INFO: `/v3/projects/b01df46ff9744c2f830459547d7e4dd6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:27.884221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01df46ff9744c2f830459547d7e4dd6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:27.887641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:27.913107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:27.953166 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01df46ff9744c2f830459547d7e4dd6', 'request_id': 'req-b3361a67-e0f7-4781-bdf4-99ab53aa602b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.962663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01df46ff9744c2f830459547d7e4dd6', 'request_id': 'req-b3361a67-e0f7-4781-bdf4-99ab53aa602b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.970328 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01df46ff9744c2f830459547d7e4dd6', 'request_id': 'req-b3361a67-e0f7-4781-bdf4-99ab53aa602b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.984903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] The token cache is being invalidate because project b01df46ff9744c2f830459547d7e4dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:42:27.985218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.985902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3361a67-e0f7-4781-bdf4-99ab53aa602b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:27.993259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d1103-21a8-40e3-bb62-1d26888573a2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:27.994216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d1103-21a8-40e3-bb62-1d26888573a2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:27.994742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d1103-21a8-40e3-bb62-1d26888573a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:27.995300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1264/2524] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:42:27 2022] DELETE /identity/v3/projects/b01df46ff9744c2f830459547d7e4dd6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:42:28.016839 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b09d1103-21a8-40e3-bb62-1d26888573a2 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:28.018259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b09d1103-21a8-40e3-bb62-1d26888573a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:28.083709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1261/2525] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:42:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:42:28.092372 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97638-8f49-46f8-814d-455c5692cefb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:28.092708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97638-8f49-46f8-814d-455c5692cefb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:28.092931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97638-8f49-46f8-814d-455c5692cefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:28.114494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e4d97638-8f49-46f8-814d-455c5692cefb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:28.115927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e4d97638-8f49-46f8-814d-455c5692cefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:28.125034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1265/2526] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:42:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:42:28.130609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d1103-21a8-40e3-bb62-1d26888573a2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:28.141452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d1103-21a8-40e3-bb62-1d26888573a2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:28.150414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4778f75-9b34-40ba-8345-bb0eb9f3acb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:28.151384 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4778f75-9b34-40ba-8345-bb0eb9f3acb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:28.151760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4778f75-9b34-40ba-8345-bb0eb9f3acb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:28.152031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4778f75-9b34-40ba-8345-bb0eb9f3acb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:28.153058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4778f75-9b34-40ba-8345-bb0eb9f3acb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:42:28.153295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4778f75-9b34-40ba-8345-bb0eb9f3acb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:28.155387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4778f75-9b34-40ba-8345-bb0eb9f3acb0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:28.183464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1262/2527] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:42:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:42:28.184722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d97638-8f49-46f8-814d-455c5692cefb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:28.198522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d97638-8f49-46f8-814d-455c5692cefb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:28.205665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c93223-740d-41a7-91bd-5e9625939143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:28.207145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c93223-740d-41a7-91bd-5e9625939143 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:28.208290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c93223-740d-41a7-91bd-5e9625939143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:28.208711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c93223-740d-41a7-91bd-5e9625939143 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:28.209738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c93223-740d-41a7-91bd-5e9625939143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:28.211694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c93223-740d-41a7-91bd-5e9625939143 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:28.247404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1266/2528] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:42:28 2022] POST /identity/v3/projects => generated 376 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:42:28.251694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4778f75-9b34-40ba-8345-bb0eb9f3acb0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:28.263736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4778f75-9b34-40ba-8345-bb0eb9f3acb0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:28.362778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8ae34-95e5-401a-aee0-c34e8f008e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:28.363475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8ae34-95e5-401a-aee0-c34e8f008e97 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:28.363877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8ae34-95e5-401a-aee0-c34e8f008e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:28.364219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8ae34-95e5-401a-aee0-c34e8f008e97 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:28.364919 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8ae34-95e5-401a-aee0-c34e8f008e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:28.366989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8ae34-95e5-401a-aee0-c34e8f008e97 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:28.393632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1263/2529] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:42:28 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:42:28.398425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c93223-740d-41a7-91bd-5e9625939143 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:28.409659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c93223-740d-41a7-91bd-5e9625939143 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:28.419452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af119b0a-e27b-4a30-b17a-59631956062d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:28.420081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af119b0a-e27b-4a30-b17a-59631956062d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:28.420362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af119b0a-e27b-4a30-b17a-59631956062d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:28.420684 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af119b0a-e27b-4a30-b17a-59631956062d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:28.421402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af119b0a-e27b-4a30-b17a-59631956062d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:28.423061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af119b0a-e27b-4a30-b17a-59631956062d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:28.439245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8ae34-95e5-401a-aee0-c34e8f008e97 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:28.440105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1267/2530] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:42:28 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:42:28.458939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8ae34-95e5-401a-aee0-c34e8f008e97 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:28.466645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b475b-fe77-4857-9819-520cfdc48820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:28.467369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b475b-fe77-4857-9819-520cfdc48820 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:28.467879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b475b-fe77-4857-9819-520cfdc48820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:28.468426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b475b-fe77-4857-9819-520cfdc48820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:28.469607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b475b-fe77-4857-9819-520cfdc48820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:28.471723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b475b-fe77-4857-9819-520cfdc48820 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:28.485358 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1264/2531] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:42:28 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:42:28.494807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af119b0a-e27b-4a30-b17a-59631956062d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:28.523716 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af119b0a-e27b-4a30-b17a-59631956062d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:28.530360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113f7a7-eb95-4aa1-86ef-b128988a5f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:28.531161 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113f7a7-eb95-4aa1-86ef-b128988a5f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:28.531541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113f7a7-eb95-4aa1-86ef-b128988a5f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:28.531923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113f7a7-eb95-4aa1-86ef-b128988a5f94 admin admin] PATH_INFO: `/v3/projects/8cdeb9d9f0eb4f1c805502bb30a99ebc/users/d8259269a28a401eb4ddbe90f128fe71/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:28.552661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113f7a7-eb95-4aa1-86ef-b128988a5f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cdeb9d9f0eb4f1c805502bb30a99ebc, user_id=d8259269a28a401eb4ddbe90f128fe71, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:28.555601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113f7a7-eb95-4aa1-86ef-b128988a5f94 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:28.578375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1268/2532] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:42:28 2022] PUT /identity/v3/projects/8cdeb9d9f0eb4f1c805502bb30a99ebc/users/d8259269a28a401eb4ddbe90f128fe71/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:42:28.582978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194a2f5-74be-4801-865a-0d85eb283ad8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:28.583482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194a2f5-74be-4801-865a-0d85eb283ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:28.583784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194a2f5-74be-4801-865a-0d85eb283ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:28.607932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4194a2f5-74be-4801-865a-0d85eb283ad8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:28.609208 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4194a2f5-74be-4801-865a-0d85eb283ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:28.679773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:28.716062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1265/2533] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:42:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:42:28.727271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113f7a7-eb95-4aa1-86ef-b128988a5f94 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:28.729058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e113f7a7-eb95-4aa1-86ef-b128988a5f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:28.795933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113f7a7-eb95-4aa1-86ef-b128988a5f94 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:28.800137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21061560-b527-442d-9dc5-6a15d8ad2530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:28.800832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21061560-b527-442d-9dc5-6a15d8ad2530 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:28.802232 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21061560-b527-442d-9dc5-6a15d8ad2530 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:28.802599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21061560-b527-442d-9dc5-6a15d8ad2530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:28.812590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21061560-b527-442d-9dc5-6a15d8ad2530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:28.814189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21061560-b527-442d-9dc5-6a15d8ad2530 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:28.830268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1269/2534] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:33.270563 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194a2f5-74be-4801-865a-0d85eb283ad8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:33.273823 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4194a2f5-74be-4801-865a-0d85eb283ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:33.291612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194a2f5-74be-4801-865a-0d85eb283ad8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:33.302462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:33.303544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:33.303829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:33.304266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] PATH_INFO: `/v3/users/c1bda5eb9abc4f6da200ea3f9500fbfa` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:33.307804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1bda5eb9abc4f6da200ea3f9500fbfa)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:33.310024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:33.376399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1bda5eb9abc4f6da200ea3f9500fbfa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:33.386864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1bda5eb9abc4f6da200ea3f9500fbfa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:33.387600 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1bda5eb9abc4f6da200ea3f9500fbfa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:33.407857 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1bda5eb9abc4f6da200ea3f9500fbfa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:33.425222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1bda5eb9abc4f6da200ea3f9500fbfa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:33.432240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1266/2535] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:42:33 2022] DELETE /identity/v3/users/c1bda5eb9abc4f6da200ea3f9500fbfa => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:42:33.734258 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21061560-b527-442d-9dc5-6a15d8ad2530 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:33.757003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21061560-b527-442d-9dc5-6a15d8ad2530 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:33.816485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:33.845231 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:33.845917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:33.846369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:33.846817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] PATH_INFO: `/v3/projects/829a41984faf4b68a3f4b1e2fd2ded27` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:33.849471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=829a41984faf4b68a3f4b1e2fd2ded27)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:33.852541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:33.952292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '829a41984faf4b68a3f4b1e2fd2ded27', 'request_id': 'req-59613982-ce29-43f9-8080-70d0e02a0ab5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:33.991710 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '829a41984faf4b68a3f4b1e2fd2ded27', 'request_id': 'req-59613982-ce29-43f9-8080-70d0e02a0ab5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:34.006439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '829a41984faf4b68a3f4b1e2fd2ded27', 'request_id': 'req-59613982-ce29-43f9-8080-70d0e02a0ab5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:34.031891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] The token cache is being invalidate because project 829a41984faf4b68a3f4b1e2fd2ded27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:42:34.032257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:34.038066 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:34.044059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1270/2536] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:42:33 2022] DELETE /identity/v3/projects/829a41984faf4b68a3f4b1e2fd2ded27 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:42:34.044757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:34.047101 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:34.107043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15876b0-7a01-4590-875d-b98eed4bed60 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:34.340066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:34.342176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:34.343449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:34.344231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] PATH_INFO: `/v3/users/1d45cdd023884945adc6783ba94fac79` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:34.347742 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d45cdd023884945adc6783ba94fac79)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:34.351578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:34.423892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d45cdd023884945adc6783ba94fac79'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:34.437320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d45cdd023884945adc6783ba94fac79'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:34.438464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d45cdd023884945adc6783ba94fac79'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:34.443696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:34.474210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d45cdd023884945adc6783ba94fac79'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:34.494699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bee441c9-dbfd-47ed-bf3f-3c3588cfaa18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d45cdd023884945adc6783ba94fac79'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:34.512112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1267/2537] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:42:34 2022] DELETE /identity/v3/users/1d45cdd023884945adc6783ba94fac79 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:42:34.836553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:34.840170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:34.874805 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59613982-ce29-43f9-8080-70d0e02a0ab5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:34.910799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6c6a-b703-44d7-a5c9-357e7b6fd48b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:34.911316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6c6a-b703-44d7-a5c9-357e7b6fd48b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:34.911767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6c6a-b703-44d7-a5c9-357e7b6fd48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:34.954324 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:34.954970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:34.955393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:34.955626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] PATH_INFO: `/v3/projects/6fb0902803ac42eda4b0e3f9292f9906` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:34.955924 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a88e6c6a-b703-44d7-a5c9-357e7b6fd48b None None] MFA Rules not processed for user `d8259269a28a401eb4ddbe90f128fe71`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:34.956916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb0902803ac42eda4b0e3f9292f9906)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:34.957807 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a88e6c6a-b703-44d7-a5c9-357e7b6fd48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:34.959181 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:35.059290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb0902803ac42eda4b0e3f9292f9906', 'request_id': 'req-67db6b0b-aea3-41de-9f08-763c314e7912'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:35.068527 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1268/2538] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:42:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:42:35.071148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb0902803ac42eda4b0e3f9292f9906', 'request_id': 'req-67db6b0b-aea3-41de-9f08-763c314e7912'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:35.078563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb0902803ac42eda4b0e3f9292f9906', 'request_id': 'req-67db6b0b-aea3-41de-9f08-763c314e7912'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:35.082769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e6c6a-b703-44d7-a5c9-357e7b6fd48b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:42:35.084525 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a88e6c6a-b703-44d7-a5c9-357e7b6fd48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:35.094317 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] The token cache is being invalidate because project 6fb0902803ac42eda4b0e3f9292f9906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:42:35.094592 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:35.095110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-67db6b0b-aea3-41de-9f08-763c314e7912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:42:35.109724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1271/2539] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:42:34 2022] DELETE /identity/v3/projects/6fb0902803ac42eda4b0e3f9292f9906 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:42:35.209366 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e6c6a-b703-44d7-a5c9-357e7b6fd48b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:42:35.281105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:35.307118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e088a-2163-4423-969c-321150faaaf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:42:35.308232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e088a-2163-4423-969c-321150faaaf0 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:35.308707 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e088a-2163-4423-969c-321150faaaf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:35.309205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e088a-2163-4423-969c-321150faaaf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:35.312016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2e9e088a-2163-4423-969c-321150faaaf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:35.399267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e088a-2163-4423-969c-321150faaaf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:42:35.401133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e088a-2163-4423-969c-321150faaaf0 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:42:35.412192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1269/2540] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:42:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:42:36.903844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35129ffb-1de6-4a05-9654-02737e9092b6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:42:36.904391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35129ffb-1de6-4a05-9654-02737e9092b6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:42:36.904391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35129ffb-1de6-4a05-9654-02737e9092b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:42:36.912913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:36.925645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-35129ffb-1de6-4a05-9654-02737e9092b6 None None] MFA Rules not processed for user `d8259269a28a401eb4ddbe90f128fe71`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:42:36.928062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-35129ffb-1de6-4a05-9654-02737e9092b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:42:36.935109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1272/2541] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:42:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:42:45.406844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=28 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:42:46.928155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:09.474190 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13d8e3-24f3-4eb8-8ff4-3973bfbe7e22 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:09.474847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13d8e3-24f3-4eb8-8ff4-3973bfbe7e22 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:09.474847 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13d8e3-24f3-4eb8-8ff4-3973bfbe7e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:09.495467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1f13d8e3-24f3-4eb8-8ff4-3973bfbe7e22 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:09.496678 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1f13d8e3-24f3-4eb8-8ff4-3973bfbe7e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:09.564934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1270/2542] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:43:09 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:09.577860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35129ffb-1de6-4a05-9654-02737e9092b6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:09.579382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-35129ffb-1de6-4a05-9654-02737e9092b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:09.663686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35129ffb-1de6-4a05-9654-02737e9092b6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:09.669989 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62a8e5-ccb8-4619-85cc-d49d084f2d22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:09.670630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62a8e5-ccb8-4619-85cc-d49d084f2d22 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:09.670913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62a8e5-ccb8-4619-85cc-d49d084f2d22 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:09.671197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62a8e5-ccb8-4619-85cc-d49d084f2d22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:09.680797 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62a8e5-ccb8-4619-85cc-d49d084f2d22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:09.682446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62a8e5-ccb8-4619-85cc-d49d084f2d22 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:09.692892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1273/2543] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:43:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:43:19.558116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=25 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:19.687793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=32 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:22.586479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718054ed-9615-4402-b580-d75ce692fbad None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:22.587069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718054ed-9615-4402-b580-d75ce692fbad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:22.587069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718054ed-9615-4402-b580-d75ce692fbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:22.609565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-718054ed-9615-4402-b580-d75ce692fbad None None] MFA Rules not processed for user `d3848643d2c44951b78814ccf7d504af`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:22.611402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-718054ed-9615-4402-b580-d75ce692fbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:22.691099 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1271/2544] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:43:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:24.753887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56019e72-6b6a-4506-bb36-2bc7556093a1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:24.755353 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56019e72-6b6a-4506-bb36-2bc7556093a1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:24.756147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56019e72-6b6a-4506-bb36-2bc7556093a1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:24.759039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1274/2545] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:43:24 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:43:24.771975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ae5af-ee4e-4b7c-a123-ad9292787143 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:24.772773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ae5af-ee4e-4b7c-a123-ad9292787143 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:24.773197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ae5af-ee4e-4b7c-a123-ad9292787143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:24.810954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-108ae5af-ee4e-4b7c-a123-ad9292787143 None None] MFA Rules not processed for user `c34f6f3e93b54a5d92b6f2f9f0de40f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:24.814075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-108ae5af-ee4e-4b7c-a123-ad9292787143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:24.849766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1272/2546] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:43:24 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:24.877291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56019e72-6b6a-4506-bb36-2bc7556093a1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:24.880625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-56019e72-6b6a-4506-bb36-2bc7556093a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:25.079202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56019e72-6b6a-4506-bb36-2bc7556093a1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:25.084186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f7e17-e91a-4625-9785-a1d08d2f790a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0891d81e6e24ad99a9b532059c8c2af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:25.085506 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f7e17-e91a-4625-9785-a1d08d2f790a service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:25.085833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f7e17-e91a-4625-9785-a1d08d2f790a service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:25.086107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f7e17-e91a-4625-9785-a1d08d2f790a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:25.098276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f7e17-e91a-4625-9785-a1d08d2f790a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:25.099738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f7e17-e91a-4625-9785-a1d08d2f790a service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:25.112493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1275/2547] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:24 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:43:25.220873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f215a-c6a1-4fcb-beb6-1604a5179a1f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:25.222589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f215a-c6a1-4fcb-beb6-1604a5179a1f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:25.223229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f215a-c6a1-4fcb-beb6-1604a5179a1f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:25.226005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1273/2548] 10.222.0.100 () {54 vars in 926 bytes} [Thu Sep 1 17:43:25 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Sep 01 17:43:25.233261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170eb66-f9fd-409d-b5db-2b9556bc9d15 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:25.233867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170eb66-f9fd-409d-b5db-2b9556bc9d15 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:25.234293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170eb66-f9fd-409d-b5db-2b9556bc9d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:25.263005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6170eb66-f9fd-409d-b5db-2b9556bc9d15 None None] MFA Rules not processed for user `bbe180b45f2e49e2abbf6e833ce0d44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:25.264633 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6170eb66-f9fd-409d-b5db-2b9556bc9d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:25.336973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1276/2549] 10.222.0.100 () {58 vars in 1023 bytes} [Thu Sep 1 17:43:25 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:43:25.352840 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f215a-c6a1-4fcb-beb6-1604a5179a1f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:25.355737 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a13f215a-c6a1-4fcb-beb6-1604a5179a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:25.384825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f215a-c6a1-4fcb-beb6-1604a5179a1f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:25.391023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498783a-979f-4b4d-b1f4-8dd73bc6a4e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:25.392238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498783a-979f-4b4d-b1f4-8dd73bc6a4e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:25.392722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498783a-979f-4b4d-b1f4-8dd73bc6a4e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:25.393352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498783a-979f-4b4d-b1f4-8dd73bc6a4e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:25.407595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498783a-979f-4b4d-b1f4-8dd73bc6a4e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:25.410327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498783a-979f-4b4d-b1f4-8dd73bc6a4e7 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:25.424947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1274/2550] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:25 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:43:28.779035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914fa8-2c1d-4d3d-94f1-f211f02428c5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:28.779888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914fa8-2c1d-4d3d-94f1-f211f02428c5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:28.779888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914fa8-2c1d-4d3d-94f1-f211f02428c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:28.807544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-50914fa8-2c1d-4d3d-94f1-f211f02428c5 None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:28.808791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-50914fa8-2c1d-4d3d-94f1-f211f02428c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:28.819169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1277/2551] 10.222.0.100 () {58 vars in 1014 bytes} [Thu Sep 1 17:43:28 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:43:28.849378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498783a-979f-4b4d-b1f4-8dd73bc6a4e7 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:28.862501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498783a-979f-4b4d-b1f4-8dd73bc6a4e7 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:28.870391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d7b72-693d-4add-a4d9-2ee40f634078 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:28.871023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d7b72-693d-4add-a4d9-2ee40f634078 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:28.871299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d7b72-693d-4add-a4d9-2ee40f634078 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:28.871563 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d7b72-693d-4add-a4d9-2ee40f634078 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:28.882469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d7b72-693d-4add-a4d9-2ee40f634078 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:28.884112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d7b72-693d-4add-a4d9-2ee40f634078 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:28.895011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1275/2552] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:43:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:43:29.402943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914fa8-2c1d-4d3d-94f1-f211f02428c5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:29.404788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-50914fa8-2c1d-4d3d-94f1-f211f02428c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:29.433733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914fa8-2c1d-4d3d-94f1-f211f02428c5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:29.441539 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7934b94-d30c-4d06-8fed-6e3f4bdf3c6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:29.442210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7934b94-d30c-4d06-8fed-6e3f4bdf3c6c service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:29.442501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7934b94-d30c-4d06-8fed-6e3f4bdf3c6c service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:29.442774 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7934b94-d30c-4d06-8fed-6e3f4bdf3c6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:29.444583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f7934b94-d30c-4d06-8fed-6e3f4bdf3c6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:29.457429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7934b94-d30c-4d06-8fed-6e3f4bdf3c6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:29.459271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7934b94-d30c-4d06-8fed-6e3f4bdf3c6c service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:29.477947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1278/2553] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:43:38.890569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=40 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:39.464736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=58 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:44.779144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad710b9b-d637-4e82-8952-89e5b4f11daa None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:44.780041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad710b9b-d637-4e82-8952-89e5b4f11daa None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:44.780041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad710b9b-d637-4e82-8952-89e5b4f11daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:44.804818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ad710b9b-d637-4e82-8952-89e5b4f11daa None None] MFA Rules not processed for user `71b9335f3f744a31a84ab278d9b45e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:44.806103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ad710b9b-d637-4e82-8952-89e5b4f11daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:44.813404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1276/2554] 10.222.0.100 () {58 vars in 972 bytes} [Thu Sep 1 17:43:44 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:43:44.825562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7934b94-d30c-4d06-8fed-6e3f4bdf3c6c service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:44.837700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7934b94-d30c-4d06-8fed-6e3f4bdf3c6c service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:44.843310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42165c7f-914e-41d7-9be3-da4d442dd7ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3fa2499f5594b7798f6fe4443a14f4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:44.843937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42165c7f-914e-41d7-9be3-da4d442dd7ef service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:44.844262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42165c7f-914e-41d7-9be3-da4d442dd7ef service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:44.844537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42165c7f-914e-41d7-9be3-da4d442dd7ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:44.854452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42165c7f-914e-41d7-9be3-da4d442dd7ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:44.856029 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42165c7f-914e-41d7-9be3-da4d442dd7ef service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:44.865913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1279/2555] 10.222.0.100 () {60 vars in 1389 bytes} [Thu Sep 1 17:43:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:43:54.798364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:54.861384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:43:59.104945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad710b9b-d637-4e82-8952-89e5b4f11daa None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:59.106695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ad710b9b-d637-4e82-8952-89e5b4f11daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:59.120363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad710b9b-d637-4e82-8952-89e5b4f11daa None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:59.124039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:59.124645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:59.124918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:59.125182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:59.127144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:59.189991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:59.192120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:59.201453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1277/2556] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:43:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:04.161954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42165c7f-914e-41d7-9be3-da4d442dd7ef service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:04.171557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42165c7f-914e-41d7-9be3-da4d442dd7ef service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:04.175485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4f9a6-8a62-4bc7-a763-044d28daeb9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:04.176323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4f9a6-8a62-4bc7-a763-044d28daeb9e service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.176597 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4f9a6-8a62-4bc7-a763-044d28daeb9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.176863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4f9a6-8a62-4bc7-a763-044d28daeb9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.178649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6eb4f9a6-8a62-4bc7-a763-044d28daeb9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:04.244227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4f9a6-8a62-4bc7-a763-044d28daeb9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:04.245697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4f9a6-8a62-4bc7-a763-044d28daeb9e service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:04.255939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1280/2557] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:09.197141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=28 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:11.169961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:11.171455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:11.186980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ffab9-9694-4816-afbd-169a8174e60d service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:11.194753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:11.195861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:11.196368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:11.196869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] PATH_INFO: `/v3/users/289058107592455fbf5af315fba7a126` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:11.198866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=289058107592455fbf5af315fba7a126)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:11.202665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:11.271921 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289058107592455fbf5af315fba7a126'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.280891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289058107592455fbf5af315fba7a126'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.281608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289058107592455fbf5af315fba7a126'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.295700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289058107592455fbf5af315fba7a126'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.303637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289058107592455fbf5af315fba7a126'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.309636 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1278/2558] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:11 2022] DELETE /identity/v3/users/289058107592455fbf5af315fba7a126 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:11.499536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4f9a6-8a62-4bc7-a763-044d28daeb9e service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:11.515676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4f9a6-8a62-4bc7-a763-044d28daeb9e service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:11.617660 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:11.618615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:11.619130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:11.619630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] PATH_INFO: `/v3/projects/b3f3bf24fbbd42919ad87cc04e4f6042` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:11.621561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f3bf24fbbd42919ad87cc04e4f6042)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:11.625304 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:11.679732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f3bf24fbbd42919ad87cc04e4f6042', 'request_id': 'req-22bed2bf-89d1-4eb0-a439-6b0a158397f8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.688017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f3bf24fbbd42919ad87cc04e4f6042', 'request_id': 'req-22bed2bf-89d1-4eb0-a439-6b0a158397f8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.694543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f3bf24fbbd42919ad87cc04e4f6042', 'request_id': 'req-22bed2bf-89d1-4eb0-a439-6b0a158397f8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.707174 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] The token cache is being invalidate because project b3f3bf24fbbd42919ad87cc04e4f6042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:11.707456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.708393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.709969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1281/2559] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:11 2022] DELETE /identity/v3/projects/b3f3bf24fbbd42919ad87cc04e4f6042 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:11.714943 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:11.717320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:11.741302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79e70e-ca81-4a48-895a-1ae49fbae3c5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:11.822734 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:11.823376 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:11.823658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:11.823952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] PATH_INFO: `/v3/users/ea59a76ca4794d3084495a5e20bb1517` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:11.825116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea59a76ca4794d3084495a5e20bb1517)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:11.827093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:11.874554 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea59a76ca4794d3084495a5e20bb1517'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.884633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea59a76ca4794d3084495a5e20bb1517'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.885261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea59a76ca4794d3084495a5e20bb1517'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.898643 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea59a76ca4794d3084495a5e20bb1517'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.900883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:11.907148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea59a76ca4794d3084495a5e20bb1517'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:11.914217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1279/2560] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:11 2022] DELETE /identity/v3/users/ea59a76ca4794d3084495a5e20bb1517 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:12.121862 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:12.123346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:12.140336 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bed2bf-89d1-4eb0-a439-6b0a158397f8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:12.164384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:12.213013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:12.213701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:12.213982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:12.214252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] PATH_INFO: `/v3/projects/0930f174a68749ad9865046cdf512fa4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:12.215862 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0930f174a68749ad9865046cdf512fa4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:12.217811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:12.275558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0930f174a68749ad9865046cdf512fa4', 'request_id': 'req-32e78977-8e9a-4cb7-abd8-d615162e1186'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.287840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0930f174a68749ad9865046cdf512fa4', 'request_id': 'req-32e78977-8e9a-4cb7-abd8-d615162e1186'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.301374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0930f174a68749ad9865046cdf512fa4', 'request_id': 'req-32e78977-8e9a-4cb7-abd8-d615162e1186'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.314371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] The token cache is being invalidate because project 0930f174a68749ad9865046cdf512fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:12.314650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.315322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:12.317401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1282/2561] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:12 2022] DELETE /identity/v3/projects/0930f174a68749ad9865046cdf512fa4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:21.912584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:22.310952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:26.304658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:26.306156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:26.369967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da197e-29f0-4e3b-94eb-a60ee169b5be admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:26.418537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4070def-2025-4d67-a836-4493200ef3ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:26.419136 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4070def-2025-4d67-a836-4493200ef3ed service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:26.419408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4070def-2025-4d67-a836-4493200ef3ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:26.419675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4070def-2025-4d67-a836-4493200ef3ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:26.421396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b4070def-2025-4d67-a836-4493200ef3ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:26.483412 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4070def-2025-4d67-a836-4493200ef3ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:26.484756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4070def-2025-4d67-a836-4493200ef3ed service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:26.509049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1280/2562] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:29.860072 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:29.861617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:29.878943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e78977-8e9a-4cb7-abd8-d615162e1186 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:29.884589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:29.885240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:29.885528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:29.885831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] PATH_INFO: `/v3/users/bcc538bc852f404699d91d17023ae620` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:29.887012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc538bc852f404699d91d17023ae620)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:29.888968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:29.941319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc538bc852f404699d91d17023ae620'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:29.954257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc538bc852f404699d91d17023ae620'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:29.961087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc538bc852f404699d91d17023ae620'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:29.968366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc538bc852f404699d91d17023ae620'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:29.968989 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-df05e209-1ce3-4b72-af52-d16f8dfbf4c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc538bc852f404699d91d17023ae620'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:29.975113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1283/2563] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:29 2022] DELETE /identity/v3/users/bcc538bc852f404699d91d17023ae620 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:30.158349 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4070def-2025-4d67-a836-4493200ef3ed service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.168905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4070def-2025-4d67-a836-4493200ef3ed service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.230967 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.231599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.231881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.232156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] PATH_INFO: `/v3/projects/e11b8d4a14a9460eb4ce9967f86e4757` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.233288 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e11b8d4a14a9460eb4ce9967f86e4757)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.235246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.244422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:30.285977 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e11b8d4a14a9460eb4ce9967f86e4757', 'request_id': 'req-902c010b-7bd8-44ce-aba9-662bcaec7a5c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:30.293299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e11b8d4a14a9460eb4ce9967f86e4757', 'request_id': 'req-902c010b-7bd8-44ce-aba9-662bcaec7a5c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:30.300465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e11b8d4a14a9460eb4ce9967f86e4757', 'request_id': 'req-902c010b-7bd8-44ce-aba9-662bcaec7a5c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:30.315814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] The token cache is being invalidate because project e11b8d4a14a9460eb4ce9967f86e4757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:30.316157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:30.316770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-902c010b-7bd8-44ce-aba9-662bcaec7a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:30.319027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1281/2564] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:30 2022] DELETE /identity/v3/projects/e11b8d4a14a9460eb4ce9967f86e4757 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:30.323971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d9e5a-cda4-450d-af18-59ad8e5f826e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.324371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d9e5a-cda4-450d-af18-59ad8e5f826e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.324732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d9e5a-cda4-450d-af18-59ad8e5f826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.345170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-547d9e5a-cda4-450d-af18-59ad8e5f826e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:30.346326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-547d9e5a-cda4-450d-af18-59ad8e5f826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:30.410641 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1284/2565] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:44:30.419806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f977c-9453-4e45-9b72-b98818ea4643 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.420084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f977c-9453-4e45-9b72-b98818ea4643 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.420367 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f977c-9453-4e45-9b72-b98818ea4643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.441282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-213f977c-9453-4e45-9b72-b98818ea4643 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:30.443116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-213f977c-9453-4e45-9b72-b98818ea4643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:30.451220 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1282/2566] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:30.455623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d9e5a-cda4-450d-af18-59ad8e5f826e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.466923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d9e5a-cda4-450d-af18-59ad8e5f826e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.471486 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2394b-6f85-436f-80b0-cef7b87b6602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.472138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2394b-6f85-436f-80b0-cef7b87b6602 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.472418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2394b-6f85-436f-80b0-cef7b87b6602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.472692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2394b-6f85-436f-80b0-cef7b87b6602 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.473335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2394b-6f85-436f-80b0-cef7b87b6602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:30.473665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2394b-6f85-436f-80b0-cef7b87b6602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.475887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2394b-6f85-436f-80b0-cef7b87b6602 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.492150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1285/2567] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:30.493282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f977c-9453-4e45-9b72-b98818ea4643 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.503436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f977c-9453-4e45-9b72-b98818ea4643 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.509342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2645008-510c-4800-9322-69ad3de29916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.510049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2645008-510c-4800-9322-69ad3de29916 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.510325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2645008-510c-4800-9322-69ad3de29916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.510591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2645008-510c-4800-9322-69ad3de29916 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.511241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2645008-510c-4800-9322-69ad3de29916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.513172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2645008-510c-4800-9322-69ad3de29916 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.567676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1283/2568] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:30 2022] POST /identity/v3/projects => generated 404 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:30.572071 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2394b-6f85-436f-80b0-cef7b87b6602 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.582217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2394b-6f85-436f-80b0-cef7b87b6602 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.639237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:30.685085 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86aeea-3658-432f-a72a-31a29533de81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.685711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86aeea-3658-432f-a72a-31a29533de81 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.685988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86aeea-3658-432f-a72a-31a29533de81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.686254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86aeea-3658-432f-a72a-31a29533de81 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.686866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86aeea-3658-432f-a72a-31a29533de81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.688801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86aeea-3658-432f-a72a-31a29533de81 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.711057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1286/2569] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:30 2022] POST /identity/v3/users => generated 303 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:30.715342 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2645008-510c-4800-9322-69ad3de29916 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.763534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2645008-510c-4800-9322-69ad3de29916 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.768214 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b67a8-ac3f-4afc-bba2-261df5e66a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.768812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b67a8-ac3f-4afc-bba2-261df5e66a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.769092 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b67a8-ac3f-4afc-bba2-261df5e66a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.769364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b67a8-ac3f-4afc-bba2-261df5e66a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.770030 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b67a8-ac3f-4afc-bba2-261df5e66a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.771656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b67a8-ac3f-4afc-bba2-261df5e66a50 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.781518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1284/2570] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/roles => generated 1427 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:44:30.785598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86aeea-3658-432f-a72a-31a29533de81 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.796211 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86aeea-3658-432f-a72a-31a29533de81 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.800802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7300e9-b1d1-440d-a53d-2a9c53032337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.801419 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7300e9-b1d1-440d-a53d-2a9c53032337 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.802258 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7300e9-b1d1-440d-a53d-2a9c53032337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.802878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7300e9-b1d1-440d-a53d-2a9c53032337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.803192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7300e9-b1d1-440d-a53d-2a9c53032337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.804786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7300e9-b1d1-440d-a53d-2a9c53032337 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.815342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1287/2571] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:44:30.820234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b67a8-ac3f-4afc-bba2-261df5e66a50 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.830194 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b67a8-ac3f-4afc-bba2-261df5e66a50 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.834884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621befb-748b-4ca5-9012-42c3ced7eddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.835535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621befb-748b-4ca5-9012-42c3ced7eddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.835812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621befb-748b-4ca5-9012-42c3ced7eddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.836085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621befb-748b-4ca5-9012-42c3ced7eddd admin admin] PATH_INFO: `/v3/projects/b2e8d07a3aa74ce384e55c2a2ce9cccc/users/dc5a1a399c5446a899e328898c9526cc/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.859116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621befb-748b-4ca5-9012-42c3ced7eddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e8d07a3aa74ce384e55c2a2ce9cccc, user_id=dc5a1a399c5446a899e328898c9526cc, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.862942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621befb-748b-4ca5-9012-42c3ced7eddd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.878283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1285/2572] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:30 2022] PUT /identity/v3/projects/b2e8d07a3aa74ce384e55c2a2ce9cccc/users/dc5a1a399c5446a899e328898c9526cc/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:30.884493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48113c80-3308-41be-9b00-dd10bdbdf458 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.884996 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48113c80-3308-41be-9b00-dd10bdbdf458 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.885116 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48113c80-3308-41be-9b00-dd10bdbdf458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.917783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-48113c80-3308-41be-9b00-dd10bdbdf458 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:30.919023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-48113c80-3308-41be-9b00-dd10bdbdf458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:30.979583 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1288/2573] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:30.990318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621befb-748b-4ca5-9012-42c3ced7eddd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.992154 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4621befb-748b-4ca5-9012-42c3ced7eddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:31.043469 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:31.077970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621befb-748b-4ca5-9012-42c3ced7eddd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:31.081637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93906cf-0c6c-4f25-88d4-d5d248f1a5d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:31.082244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93906cf-0c6c-4f25-88d4-d5d248f1a5d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.082526 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93906cf-0c6c-4f25-88d4-d5d248f1a5d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.082911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93906cf-0c6c-4f25-88d4-d5d248f1a5d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.092738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93906cf-0c6c-4f25-88d4-d5d248f1a5d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:31.094207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93906cf-0c6c-4f25-88d4-d5d248f1a5d6 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:31.104721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1286/2574] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:44:31.810352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c0b16-d93c-4134-9056-9cecf6c4897a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.810637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c0b16-d93c-4134-9056-9cecf6c4897a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.810911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c0b16-d93c-4134-9056-9cecf6c4897a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.890484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c80c0b16-d93c-4134-9056-9cecf6c4897a None None] MFA Rules not processed for user `dc5a1a399c5446a899e328898c9526cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:31.892981 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c80c0b16-d93c-4134-9056-9cecf6c4897a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:32.031595 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1289/2575] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:31 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:32.036377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93906cf-0c6c-4f25-88d4-d5d248f1a5d6 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.048864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93906cf-0c6c-4f25-88d4-d5d248f1a5d6 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.056871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4b280-cec7-4167-8c54-38a7199142d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.059415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4b280-cec7-4167-8c54-38a7199142d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.059796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4b280-cec7-4167-8c54-38a7199142d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.060140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4b280-cec7-4167-8c54-38a7199142d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.061370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4b280-cec7-4167-8c54-38a7199142d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.063021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4b280-cec7-4167-8c54-38a7199142d3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.101288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c0b16-d93c-4134-9056-9cecf6c4897a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.102898 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1287/2576] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:32 2022] POST /identity/v3/projects => generated 406 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:32.122252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c0b16-d93c-4134-9056-9cecf6c4897a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.154023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:32.201847 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ed938-f399-410f-ae5f-769ad58c4e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.202497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ed938-f399-410f-ae5f-769ad58c4e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.202779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ed938-f399-410f-ae5f-769ad58c4e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.203055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ed938-f399-410f-ae5f-769ad58c4e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.203688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ed938-f399-410f-ae5f-769ad58c4e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.205642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ed938-f399-410f-ae5f-769ad58c4e8c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.230536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1290/2577] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:32 2022] POST /identity/v3/users => generated 310 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:32.231610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4b280-cec7-4167-8c54-38a7199142d3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.243198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4b280-cec7-4167-8c54-38a7199142d3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.250290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da95df-ebd0-464d-b04a-0da468050c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.251050 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da95df-ebd0-464d-b04a-0da468050c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.251461 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da95df-ebd0-464d-b04a-0da468050c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.251798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da95df-ebd0-464d-b04a-0da468050c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.252575 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da95df-ebd0-464d-b04a-0da468050c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.254300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da95df-ebd0-464d-b04a-0da468050c3f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.267955 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1288/2578] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:32 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:32.272582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ed938-f399-410f-ae5f-769ad58c4e8c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.282697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ed938-f399-410f-ae5f-769ad58c4e8c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.289014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06253483-1a6b-484e-9b6e-5b69dafc7ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.289235 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253483-1a6b-484e-9b6e-5b69dafc7ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.289401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253483-1a6b-484e-9b6e-5b69dafc7ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.289723 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253483-1a6b-484e-9b6e-5b69dafc7ea9 admin admin] PATH_INFO: `/v3/projects/8492edd831f54d5283a50a1a1cdd319d/users/fee0c8e70c1a400cbc2ec646cd423e74/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.307887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06253483-1a6b-484e-9b6e-5b69dafc7ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8492edd831f54d5283a50a1a1cdd319d, user_id=fee0c8e70c1a400cbc2ec646cd423e74, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.310623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06253483-1a6b-484e-9b6e-5b69dafc7ea9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.324462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1291/2579] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:32 2022] PUT /identity/v3/projects/8492edd831f54d5283a50a1a1cdd319d/users/fee0c8e70c1a400cbc2ec646cd423e74/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:32.329134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da95df-ebd0-464d-b04a-0da468050c3f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.339256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da95df-ebd0-464d-b04a-0da468050c3f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.399548 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2a3ea-84ad-4cdd-8a5c-e5dfbcafd6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.400162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2a3ea-84ad-4cdd-8a5c-e5dfbcafd6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.400442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2a3ea-84ad-4cdd-8a5c-e5dfbcafd6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.400715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2a3ea-84ad-4cdd-8a5c-e5dfbcafd6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.401354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2a3ea-84ad-4cdd-8a5c-e5dfbcafd6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.403001 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2a3ea-84ad-4cdd-8a5c-e5dfbcafd6d5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.414102 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1289/2580] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:32 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:32.418834 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06253483-1a6b-484e-9b6e-5b69dafc7ea9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.430192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06253483-1a6b-484e-9b6e-5b69dafc7ea9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.435454 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46925f-2fc3-44a3-9e95-809d412ec187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.436234 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46925f-2fc3-44a3-9e95-809d412ec187 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.436576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46925f-2fc3-44a3-9e95-809d412ec187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.436912 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46925f-2fc3-44a3-9e95-809d412ec187 admin admin] PATH_INFO: `/v3/projects/8492edd831f54d5283a50a1a1cdd319d/users/fee0c8e70c1a400cbc2ec646cd423e74/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.439027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46925f-2fc3-44a3-9e95-809d412ec187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8492edd831f54d5283a50a1a1cdd319d, user_id=fee0c8e70c1a400cbc2ec646cd423e74, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.444363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46925f-2fc3-44a3-9e95-809d412ec187 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.457253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1292/2581] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:32 2022] PUT /identity/v3/projects/8492edd831f54d5283a50a1a1cdd319d/users/fee0c8e70c1a400cbc2ec646cd423e74/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:33.293347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0871ca-9dbc-4e5b-8970-32cb72094e6f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:33.293859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0871ca-9dbc-4e5b-8970-32cb72094e6f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:33.293980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0871ca-9dbc-4e5b-8970-32cb72094e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:33.325443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8d0871ca-9dbc-4e5b-8970-32cb72094e6f None None] MFA Rules not processed for user `fee0c8e70c1a400cbc2ec646cd423e74`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:33.326494 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8d0871ca-9dbc-4e5b-8970-32cb72094e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:33.406451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1290/2582] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:33 2022] POST /identity/v3/auth/tokens => generated 3359 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:34.424283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46925f-2fc3-44a3-9e95-809d412ec187 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:34.426142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-be46925f-2fc3-44a3-9e95-809d412ec187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:34.490689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0871ca-9dbc-4e5b-8970-32cb72094e6f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:34.514927 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:34.531163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46925f-2fc3-44a3-9e95-809d412ec187 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:34.537445 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4c75a-3bde-49dd-b4a7-8bbeaf5bc13e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:34.538087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4c75a-3bde-49dd-b4a7-8bbeaf5bc13e service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.538377 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4c75a-3bde-49dd-b4a7-8bbeaf5bc13e service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.538655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4c75a-3bde-49dd-b4a7-8bbeaf5bc13e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.540442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-85b4c75a-3bde-49dd-b4a7-8bbeaf5bc13e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:34.552768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4c75a-3bde-49dd-b4a7-8bbeaf5bc13e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:34.554386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4c75a-3bde-49dd-b4a7-8bbeaf5bc13e service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:34.564764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1293/2583] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:34 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:34.565399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0871ca-9dbc-4e5b-8970-32cb72094e6f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:34.570115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:34.570776 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.571052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.571620 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] PATH_INFO: `/v3/users/dc5a1a399c5446a899e328898c9526cc` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.572821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5a1a399c5446a899e328898c9526cc)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:34.574775 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:34.629873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5a1a399c5446a899e328898c9526cc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:34.637672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5a1a399c5446a899e328898c9526cc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:34.638346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5a1a399c5446a899e328898c9526cc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:34.651178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5a1a399c5446a899e328898c9526cc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:34.658546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5a1a399c5446a899e328898c9526cc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:34.665956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1291/2584] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:34 2022] DELETE /identity/v3/users/dc5a1a399c5446a899e328898c9526cc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:34.855886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4c75a-3bde-49dd-b4a7-8bbeaf5bc13e service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:34.872208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4c75a-3bde-49dd-b4a7-8bbeaf5bc13e service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:34.943747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:34.963451 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:34.964233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.964572 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.965057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] PATH_INFO: `/v3/projects/b2e8d07a3aa74ce384e55c2a2ce9cccc` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.966274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e8d07a3aa74ce384e55c2a2ce9cccc)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:34.968404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.019287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e8d07a3aa74ce384e55c2a2ce9cccc', 'request_id': 'req-828e5f6c-d54f-441d-b86d-a329c77196b7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.028544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e8d07a3aa74ce384e55c2a2ce9cccc', 'request_id': 'req-828e5f6c-d54f-441d-b86d-a329c77196b7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.035343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e8d07a3aa74ce384e55c2a2ce9cccc', 'request_id': 'req-828e5f6c-d54f-441d-b86d-a329c77196b7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.052430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] The token cache is being invalidate because project b2e8d07a3aa74ce384e55c2a2ce9cccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:35.052730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.053634 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.055584 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1294/2585] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:34 2022] DELETE /identity/v3/projects/b2e8d07a3aa74ce384e55c2a2ce9cccc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:35.059765 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.061452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:35.077262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a3a5b-44a8-4193-8d52-af4e2897f842 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.142860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.143532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.143813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.144081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] PATH_INFO: `/v3/users/fee0c8e70c1a400cbc2ec646cd423e74` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.145244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fee0c8e70c1a400cbc2ec646cd423e74)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.147215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.186553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.196094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fee0c8e70c1a400cbc2ec646cd423e74'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.204077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fee0c8e70c1a400cbc2ec646cd423e74'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.204907 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fee0c8e70c1a400cbc2ec646cd423e74'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.218221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fee0c8e70c1a400cbc2ec646cd423e74'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.226816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6df88114-e865-4a46-baad-faca20b52775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fee0c8e70c1a400cbc2ec646cd423e74'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.232748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1292/2586] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:35 2022] DELETE /identity/v3/users/fee0c8e70c1a400cbc2ec646cd423e74 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:35.406453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.408000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:35.422063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e5f6c-d54f-441d-b86d-a329c77196b7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.479513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.480244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.480406 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.480680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] PATH_INFO: `/v3/projects/8492edd831f54d5283a50a1a1cdd319d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.481878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8492edd831f54d5283a50a1a1cdd319d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.483768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.532857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8492edd831f54d5283a50a1a1cdd319d', 'request_id': 'req-6d5d0f5d-0735-4393-9b5e-a899afd3544d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.541322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8492edd831f54d5283a50a1a1cdd319d', 'request_id': 'req-6d5d0f5d-0735-4393-9b5e-a899afd3544d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.547054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8492edd831f54d5283a50a1a1cdd319d', 'request_id': 'req-6d5d0f5d-0735-4393-9b5e-a899afd3544d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.558223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] The token cache is being invalidate because project 8492edd831f54d5283a50a1a1cdd319d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:35.558512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.559152 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6d5d0f5d-0735-4393-9b5e-a899afd3544d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:35.560746 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1295/2587] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:35 2022] DELETE /identity/v3/projects/8492edd831f54d5283a50a1a1cdd319d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:35.565991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d63e6-a88e-4fee-a1e2-e3b1baa7591c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.566614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d63e6-a88e-4fee-a1e2-e3b1baa7591c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.566614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d63e6-a88e-4fee-a1e2-e3b1baa7591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.619216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a26d63e6-a88e-4fee-a1e2-e3b1baa7591c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:35.620460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a26d63e6-a88e-4fee-a1e2-e3b1baa7591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:35.682189 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1293/2588] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:35.690285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5f1f2-00d2-42c6-b557-9a2a665c1db0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.690576 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5f1f2-00d2-42c6-b557-9a2a665c1db0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.690857 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5f1f2-00d2-42c6-b557-9a2a665c1db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.699401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.716061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-31c5f1f2-00d2-42c6-b557-9a2a665c1db0 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:35.717347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-31c5f1f2-00d2-42c6-b557-9a2a665c1db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:35.726290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1296/2589] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:35.731753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d63e6-a88e-4fee-a1e2-e3b1baa7591c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.742233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d63e6-a88e-4fee-a1e2-e3b1baa7591c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.747023 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd8559-80e9-40c1-adc0-c0dcc71b75fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.747645 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd8559-80e9-40c1-adc0-c0dcc71b75fa admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.747929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd8559-80e9-40c1-adc0-c0dcc71b75fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.748209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd8559-80e9-40c1-adc0-c0dcc71b75fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.748878 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd8559-80e9-40c1-adc0-c0dcc71b75fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:35.749184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd8559-80e9-40c1-adc0-c0dcc71b75fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.750903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd8559-80e9-40c1-adc0-c0dcc71b75fa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.764146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1294/2590] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:35.768519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5f1f2-00d2-42c6-b557-9a2a665c1db0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.778836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5f1f2-00d2-42c6-b557-9a2a665c1db0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.783523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30f4b4-e677-465d-b07d-bf66b8d6958c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.784048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30f4b4-e677-465d-b07d-bf66b8d6958c admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.784323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30f4b4-e677-465d-b07d-bf66b8d6958c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.784589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30f4b4-e677-465d-b07d-bf66b8d6958c admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.785203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30f4b4-e677-465d-b07d-bf66b8d6958c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.787463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30f4b4-e677-465d-b07d-bf66b8d6958c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.816160 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1297/2591] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:35 2022] POST /identity/v3/projects => generated 382 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:35.823241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd8559-80e9-40c1-adc0-c0dcc71b75fa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.834485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd8559-80e9-40c1-adc0-c0dcc71b75fa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.892513 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e79e9-f1eb-4495-a62e-dc20139d482a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.893117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e79e9-f1eb-4495-a62e-dc20139d482a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.893402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e79e9-f1eb-4495-a62e-dc20139d482a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.893717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e79e9-f1eb-4495-a62e-dc20139d482a admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.894345 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e79e9-f1eb-4495-a62e-dc20139d482a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.897070 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e79e9-f1eb-4495-a62e-dc20139d482a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.918881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1295/2592] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:35 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:35.923028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30f4b4-e677-465d-b07d-bf66b8d6958c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.933947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30f4b4-e677-465d-b07d-bf66b8d6958c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.940278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb5f7e-8572-4b1c-b1c5-3619273abe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.940497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb5f7e-8572-4b1c-b1c5-3619273abe21 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.940777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb5f7e-8572-4b1c-b1c5-3619273abe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.941050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb5f7e-8572-4b1c-b1c5-3619273abe21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.941734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb5f7e-8572-4b1c-b1c5-3619273abe21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.944169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb5f7e-8572-4b1c-b1c5-3619273abe21 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.955689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1298/2593] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:35.960160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e79e9-f1eb-4495-a62e-dc20139d482a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.972352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e79e9-f1eb-4495-a62e-dc20139d482a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.977639 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871609c6-6c77-470e-9526-39b9de8515c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.978201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871609c6-6c77-470e-9526-39b9de8515c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.978542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871609c6-6c77-470e-9526-39b9de8515c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.978819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871609c6-6c77-470e-9526-39b9de8515c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.979646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871609c6-6c77-470e-9526-39b9de8515c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.981155 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871609c6-6c77-470e-9526-39b9de8515c1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.982613 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:36.001609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1296/2594] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:36.003837 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb5f7e-8572-4b1c-b1c5-3619273abe21 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.014580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb5f7e-8572-4b1c-b1c5-3619273abe21 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.019432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295cf42-d505-4d3b-8c12-c610051577f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.020086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295cf42-d505-4d3b-8c12-c610051577f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.020365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295cf42-d505-4d3b-8c12-c610051577f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.020649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295cf42-d505-4d3b-8c12-c610051577f4 admin admin] PATH_INFO: `/v3/projects/60a8569abcc24271ae41b6ffc85d9d4e/users/bd7d5c724af94ef09815e9c89ab436d9/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.035791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295cf42-d505-4d3b-8c12-c610051577f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a8569abcc24271ae41b6ffc85d9d4e, user_id=bd7d5c724af94ef09815e9c89ab436d9, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.038615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295cf42-d505-4d3b-8c12-c610051577f4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.053322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1299/2595] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:36 2022] PUT /identity/v3/projects/60a8569abcc24271ae41b6ffc85d9d4e/users/bd7d5c724af94ef09815e9c89ab436d9/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:36.066178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023353e-ad74-46d3-883a-b2d7bd4204ae None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.066178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023353e-ad74-46d3-883a-b2d7bd4204ae None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.066178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023353e-ad74-46d3-883a-b2d7bd4204ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.096583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e023353e-ad74-46d3-883a-b2d7bd4204ae None None] MFA Rules not processed for user `bd7d5c724af94ef09815e9c89ab436d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:36.098096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e023353e-ad74-46d3-883a-b2d7bd4204ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:36.180104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1297/2596] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:36.186303 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295cf42-d505-4d3b-8c12-c610051577f4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.252326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295cf42-d505-4d3b-8c12-c610051577f4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.258139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fec8a1-a2e1-4cdf-b2de-4bc7a7967902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.258744 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fec8a1-a2e1-4cdf-b2de-4bc7a7967902 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.259024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fec8a1-a2e1-4cdf-b2de-4bc7a7967902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.259295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fec8a1-a2e1-4cdf-b2de-4bc7a7967902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.259922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fec8a1-a2e1-4cdf-b2de-4bc7a7967902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.261888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fec8a1-a2e1-4cdf-b2de-4bc7a7967902 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.288458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1300/2597] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:36 2022] POST /identity/v3/projects => generated 382 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:36.292756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023353e-ad74-46d3-883a-b2d7bd4204ae None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.305344 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023353e-ad74-46d3-883a-b2d7bd4204ae None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.373780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58df17-ee06-4697-9170-e067c9df4dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.374388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58df17-ee06-4697-9170-e067c9df4dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.374670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58df17-ee06-4697-9170-e067c9df4dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.374945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58df17-ee06-4697-9170-e067c9df4dcb admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.375588 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58df17-ee06-4697-9170-e067c9df4dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.377504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58df17-ee06-4697-9170-e067c9df4dcb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.406279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1298/2598] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:36 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:36.410438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fec8a1-a2e1-4cdf-b2de-4bc7a7967902 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.427948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fec8a1-a2e1-4cdf-b2de-4bc7a7967902 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.433283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e38772-a934-4aea-a3d3-4d694ecafb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.433951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e38772-a934-4aea-a3d3-4d694ecafb91 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.434267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e38772-a934-4aea-a3d3-4d694ecafb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.434541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e38772-a934-4aea-a3d3-4d694ecafb91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.435185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e38772-a934-4aea-a3d3-4d694ecafb91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.436778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e38772-a934-4aea-a3d3-4d694ecafb91 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.447940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1301/2599] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:36 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:36.452142 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58df17-ee06-4697-9170-e067c9df4dcb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.462474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58df17-ee06-4697-9170-e067c9df4dcb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.467543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dd3e8-41b4-4c76-93b1-7db7e88a883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.468197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dd3e8-41b4-4c76-93b1-7db7e88a883e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.468479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dd3e8-41b4-4c76-93b1-7db7e88a883e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.468754 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dd3e8-41b4-4c76-93b1-7db7e88a883e admin admin] PATH_INFO: `/v3/projects/3eaa18cc7081479fa168c0fe2a748872/users/c4960e430eea44ada9f959d6f8d30b70/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.484644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54dd3e8-41b4-4c76-93b1-7db7e88a883e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eaa18cc7081479fa168c0fe2a748872, user_id=c4960e430eea44ada9f959d6f8d30b70, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.487300 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54dd3e8-41b4-4c76-93b1-7db7e88a883e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.500883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1299/2600] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:36 2022] PUT /identity/v3/projects/3eaa18cc7081479fa168c0fe2a748872/users/c4960e430eea44ada9f959d6f8d30b70/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:36.505426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e38772-a934-4aea-a3d3-4d694ecafb91 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.518312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e38772-a934-4aea-a3d3-4d694ecafb91 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.575623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa34edc-03e8-4e63-9c0c-4ce90fd0dd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.576177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa34edc-03e8-4e63-9c0c-4ce90fd0dd54 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.576450 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa34edc-03e8-4e63-9c0c-4ce90fd0dd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.576717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa34edc-03e8-4e63-9c0c-4ce90fd0dd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.577344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa34edc-03e8-4e63-9c0c-4ce90fd0dd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.579128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa34edc-03e8-4e63-9c0c-4ce90fd0dd54 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.589078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1302/2601] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:36 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:36.593745 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dd3e8-41b4-4c76-93b1-7db7e88a883e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.605149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dd3e8-41b4-4c76-93b1-7db7e88a883e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.609818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d38862-4a02-4516-9a3d-5ea44ba1b896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.610465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d38862-4a02-4516-9a3d-5ea44ba1b896 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.610741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d38862-4a02-4516-9a3d-5ea44ba1b896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.611011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d38862-4a02-4516-9a3d-5ea44ba1b896 admin admin] PATH_INFO: `/v3/projects/3eaa18cc7081479fa168c0fe2a748872/users/c4960e430eea44ada9f959d6f8d30b70/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.612897 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d38862-4a02-4516-9a3d-5ea44ba1b896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eaa18cc7081479fa168c0fe2a748872, user_id=c4960e430eea44ada9f959d6f8d30b70, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.616228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d38862-4a02-4516-9a3d-5ea44ba1b896 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.626454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1300/2602] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:36 2022] PUT /identity/v3/projects/3eaa18cc7081479fa168c0fe2a748872/users/c4960e430eea44ada9f959d6f8d30b70/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:36.636753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63879dde-863f-404f-9588-73834f20c703 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.637039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63879dde-863f-404f-9588-73834f20c703 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.637323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63879dde-863f-404f-9588-73834f20c703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.658103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:36.673816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-63879dde-863f-404f-9588-73834f20c703 None None] MFA Rules not processed for user `c4960e430eea44ada9f959d6f8d30b70`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:36.675052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-63879dde-863f-404f-9588-73834f20c703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:36.746440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1303/2603] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:36 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:36.759993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d38862-4a02-4516-9a3d-5ea44ba1b896 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.761818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-83d38862-4a02-4516-9a3d-5ea44ba1b896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:36.809021 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:36.827178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d38862-4a02-4516-9a3d-5ea44ba1b896 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.830819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01039c1-a707-4a7e-a8c2-d7b0a19e9163 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.831438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01039c1-a707-4a7e-a8c2-d7b0a19e9163 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.831718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01039c1-a707-4a7e-a8c2-d7b0a19e9163 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.831989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01039c1-a707-4a7e-a8c2-d7b0a19e9163 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.842818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01039c1-a707-4a7e-a8c2-d7b0a19e9163 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.844312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01039c1-a707-4a7e-a8c2-d7b0a19e9163 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.855702 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1301/2604] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:36 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:37.175679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63879dde-863f-404f-9588-73834f20c703 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:37.185513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63879dde-863f-404f-9588-73834f20c703 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:37.190132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19663f16-b77c-4b52-97b4-f5149460d5b2 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4960e430eea44ada9f959d6f8d30b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3eaa18cc7081479fa168c0fe2a748872', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:37.190729 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19663f16-b77c-4b52-97b4-f5149460d5b2 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.191012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19663f16-b77c-4b52-97b4-f5149460d5b2 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.191288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19663f16-b77c-4b52-97b4-f5149460d5b2 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.191934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19663f16-b77c-4b52-97b4-f5149460d5b2 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:37.192811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19663f16-b77c-4b52-97b4-f5149460d5b2 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:37.194561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19663f16-b77c-4b52-97b4-f5149460d5b2 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:37.206209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1304/2605] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:37.211074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01039c1-a707-4a7e-a8c2-d7b0a19e9163 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:37.220309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01039c1-a707-4a7e-a8c2-d7b0a19e9163 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:37.225335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732e64a-8c4c-4035-9db9-f1afd42325d6 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4960e430eea44ada9f959d6f8d30b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3eaa18cc7081479fa168c0fe2a748872', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:37.226019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732e64a-8c4c-4035-9db9-f1afd42325d6 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.226458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732e64a-8c4c-4035-9db9-f1afd42325d6 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.226826 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732e64a-8c4c-4035-9db9-f1afd42325d6 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.227405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732e64a-8c4c-4035-9db9-f1afd42325d6 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:37.229276 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732e64a-8c4c-4035-9db9-f1afd42325d6 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:37.261629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1302/2606] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:37 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:37.263239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19663f16-b77c-4b52-97b4-f5149460d5b2 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:37.274882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19663f16-b77c-4b52-97b4-f5149460d5b2 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:37.331728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732e64a-8c4c-4035-9db9-f1afd42325d6 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:37.333535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8732e64a-8c4c-4035-9db9-f1afd42325d6 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:37.336442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca25e-6937-4c9c-a3f5-d0a8f34ec794 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4960e430eea44ada9f959d6f8d30b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3eaa18cc7081479fa168c0fe2a748872', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:37.337056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca25e-6937-4c9c-a3f5-d0a8f34ec794 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.337338 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca25e-6937-4c9c-a3f5-d0a8f34ec794 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.337683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca25e-6937-4c9c-a3f5-d0a8f34ec794 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.338331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ca25e-6937-4c9c-a3f5-d0a8f34ec794 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:37.338856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ca25e-6937-4c9c-a3f5-d0a8f34ec794 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:37.340467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ca25e-6937-4c9c-a3f5-d0a8f34ec794 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:37.353589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1305/2607] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:37.391571 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca25e-6937-4c9c-a3f5-d0a8f34ec794 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:37.393036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732e64a-8c4c-4035-9db9-f1afd42325d6 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:37.397790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181b67e-2da4-4606-9cb4-c5b108426478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:37.398524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181b67e-2da4-4606-9cb4-c5b108426478 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.398805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181b67e-2da4-4606-9cb4-c5b108426478 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.399078 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181b67e-2da4-4606-9cb4-c5b108426478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.400820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a181b67e-2da4-4606-9cb4-c5b108426478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:37.402320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca25e-6937-4c9c-a3f5-d0a8f34ec794 tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:37.406965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4960e430eea44ada9f959d6f8d30b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3eaa18cc7081479fa168c0fe2a748872', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:37.407580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.407863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.408135 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] PATH_INFO: `/v3/projects/35c35f7ba1584fe0934200febd438f5e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.409260 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Authorizing `identity:delete_project(project_id=35c35f7ba1584fe0934200febd438f5e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:37.411262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:37.461411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c35f7ba1584fe0934200febd438f5e', 'request_id': 'req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.470535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c35f7ba1584fe0934200febd438f5e', 'request_id': 'req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.476365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181b67e-2da4-4606-9cb4-c5b108426478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:37.476563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c35f7ba1584fe0934200febd438f5e', 'request_id': 'req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.477838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181b67e-2da4-4606-9cb4-c5b108426478 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:37.484330 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:37.488219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1303/2608] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:44:37.489793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] The token cache is being invalidate because project 35c35f7ba1584fe0934200febd438f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:37.490195 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.491208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d312b8b8-f3a1-49ee-85b0-44c20f41af7d tempest-VolumeQuotaClassesTest-1067622973 tempest-VolumeQuotaClassesTest-1067622973-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.493679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1306/2609] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:37 2022] DELETE /identity/v3/projects/35c35f7ba1584fe0934200febd438f5e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:37.712391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181b67e-2da4-4606-9cb4-c5b108426478 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:37.723131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181b67e-2da4-4606-9cb4-c5b108426478 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:37.783997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:37.784630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.784908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.785275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] PATH_INFO: `/v3/users/bd7d5c724af94ef09815e9c89ab436d9` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.787038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7d5c724af94ef09815e9c89ab436d9)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:37.788885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:37.800370 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:37.840451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7d5c724af94ef09815e9c89ab436d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.850232 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7d5c724af94ef09815e9c89ab436d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.850877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7d5c724af94ef09815e9c89ab436d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.866545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7d5c724af94ef09815e9c89ab436d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.876320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7d5c724af94ef09815e9c89ab436d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:37.882421 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1304/2610] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:37 2022] DELETE /identity/v3/users/bd7d5c724af94ef09815e9c89ab436d9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:37.887051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d95a5-8132-4228-b95e-45179f7eeb71 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.887587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d95a5-8132-4228-b95e-45179f7eeb71 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.887587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d95a5-8132-4228-b95e-45179f7eeb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.906601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cd0d95a5-8132-4228-b95e-45179f7eeb71 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:37.907820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cd0d95a5-8132-4228-b95e-45179f7eeb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:37.974673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1307/2611] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:37.982442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:37.982997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:38.044632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17ec1e-3d55-4df1-8a69-c84053dae774 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.048646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451b26d-844d-4bc0-9418-640fc680c6d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.049262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451b26d-844d-4bc0-9418-640fc680c6d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.049544 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451b26d-844d-4bc0-9418-640fc680c6d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.049920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451b26d-844d-4bc0-9418-640fc680c6d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.059604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451b26d-844d-4bc0-9418-640fc680c6d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:38.061061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451b26d-844d-4bc0-9418-640fc680c6d0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:38.071518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1305/2612] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:38.354312 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d95a5-8132-4228-b95e-45179f7eeb71 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:38.355840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cd0d95a5-8132-4228-b95e-45179f7eeb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:38.369907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d95a5-8132-4228-b95e-45179f7eeb71 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.375000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.375626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.375969 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.376332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] PATH_INFO: `/v3/projects/60a8569abcc24271ae41b6ffc85d9d4e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.377530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a8569abcc24271ae41b6ffc85d9d4e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:38.379695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:38.443978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a8569abcc24271ae41b6ffc85d9d4e', 'request_id': 'req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.452391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a8569abcc24271ae41b6ffc85d9d4e', 'request_id': 'req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.459524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a8569abcc24271ae41b6ffc85d9d4e', 'request_id': 'req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.473384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] The token cache is being invalidate because project 60a8569abcc24271ae41b6ffc85d9d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:38.473800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.474431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.476225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1308/2613] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:38 2022] DELETE /identity/v3/projects/60a8569abcc24271ae41b6ffc85d9d4e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:38.479920 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451b26d-844d-4bc0-9418-640fc680c6d0 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:38.495255 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451b26d-844d-4bc0-9418-640fc680c6d0 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.586219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.586489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.587036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.587315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] PATH_INFO: `/v3/users/c4960e430eea44ada9f959d6f8d30b70` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.588684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4960e430eea44ada9f959d6f8d30b70)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:38.590812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:38.639381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:38.661881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4960e430eea44ada9f959d6f8d30b70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.671835 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4960e430eea44ada9f959d6f8d30b70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.672895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4960e430eea44ada9f959d6f8d30b70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.693947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4960e430eea44ada9f959d6f8d30b70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.701997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6a11084-a493-4026-a8ee-703d2d544ac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4960e430eea44ada9f959d6f8d30b70'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:38.709317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1306/2614] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:38 2022] DELETE /identity/v3/users/c4960e430eea44ada9f959d6f8d30b70 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:39.025387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.027500 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:39.051663 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d05fb-ef1d-43af-8e28-eb2cbdc55a39 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.157128 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.157972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.158249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.158520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] PATH_INFO: `/v3/projects/3eaa18cc7081479fa168c0fe2a748872` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.159678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eaa18cc7081479fa168c0fe2a748872)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.162790 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.210365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:39.247580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eaa18cc7081479fa168c0fe2a748872', 'request_id': 'req-297f69c6-9151-46b9-83d7-12f85cc47774'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:39.258610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eaa18cc7081479fa168c0fe2a748872', 'request_id': 'req-297f69c6-9151-46b9-83d7-12f85cc47774'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:39.267405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eaa18cc7081479fa168c0fe2a748872', 'request_id': 'req-297f69c6-9151-46b9-83d7-12f85cc47774'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:39.281099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] The token cache is being invalidate because project 3eaa18cc7081479fa168c0fe2a748872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:39.281417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:39.282087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-297f69c6-9151-46b9-83d7-12f85cc47774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:39.283804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1309/2615] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:39 2022] DELETE /identity/v3/projects/3eaa18cc7081479fa168c0fe2a748872 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:39.289145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f394e-4610-4501-a61d-b690a221307f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.289691 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f394e-4610-4501-a61d-b690a221307f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.289796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f394e-4610-4501-a61d-b690a221307f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.312877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0b2f394e-4610-4501-a61d-b690a221307f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:39.314277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0b2f394e-4610-4501-a61d-b690a221307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:39.383373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1307/2616] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:39.391300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f75da4-0c0d-4d5b-885f-154de72f2f21 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.391649 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f75da4-0c0d-4d5b-885f-154de72f2f21 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.391850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f75da4-0c0d-4d5b-885f-154de72f2f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.413698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f5f75da4-0c0d-4d5b-885f-154de72f2f21 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:39.414933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f5f75da4-0c0d-4d5b-885f-154de72f2f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:39.423527 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1310/2617] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:39.427874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f394e-4610-4501-a61d-b690a221307f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.438215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f394e-4610-4501-a61d-b690a221307f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.443650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f9b52-3eba-4f6b-83ac-97b52a1f6049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.444274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f9b52-3eba-4f6b-83ac-97b52a1f6049 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.444531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f9b52-3eba-4f6b-83ac-97b52a1f6049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.444803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f9b52-3eba-4f6b-83ac-97b52a1f6049 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.445463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f9b52-3eba-4f6b-83ac-97b52a1f6049 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:39.445794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f9b52-3eba-4f6b-83ac-97b52a1f6049 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.447663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f9b52-3eba-4f6b-83ac-97b52a1f6049 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.465339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1308/2618] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:39.466494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f75da4-0c0d-4d5b-885f-154de72f2f21 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.477108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f75da4-0c0d-4d5b-885f-154de72f2f21 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.482442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef2cc0-b195-4126-8af3-a93376a88b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.483067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef2cc0-b195-4126-8af3-a93376a88b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.483348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef2cc0-b195-4126-8af3-a93376a88b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.483621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef2cc0-b195-4126-8af3-a93376a88b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.484313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef2cc0-b195-4126-8af3-a93376a88b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.486283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef2cc0-b195-4126-8af3-a93376a88b7f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.524645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1311/2619] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:39 2022] POST /identity/v3/projects => generated 382 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:39.526638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f9b52-3eba-4f6b-83ac-97b52a1f6049 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.541198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f9b52-3eba-4f6b-83ac-97b52a1f6049 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.617074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31532a-6400-4488-85f3-e290c263bb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.617761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31532a-6400-4488-85f3-e290c263bb46 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.618047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31532a-6400-4488-85f3-e290c263bb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.618322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31532a-6400-4488-85f3-e290c263bb46 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.618960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31532a-6400-4488-85f3-e290c263bb46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.620951 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31532a-6400-4488-85f3-e290c263bb46 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.647743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1309/2620] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:39 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:39.650038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef2cc0-b195-4126-8af3-a93376a88b7f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.661192 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef2cc0-b195-4126-8af3-a93376a88b7f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.666984 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6afdc4-1e02-4545-9911-72cb3ffd1a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.667631 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6afdc4-1e02-4545-9911-72cb3ffd1a31 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.667912 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6afdc4-1e02-4545-9911-72cb3ffd1a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.668196 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6afdc4-1e02-4545-9911-72cb3ffd1a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.668906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6afdc4-1e02-4545-9911-72cb3ffd1a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.670548 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6afdc4-1e02-4545-9911-72cb3ffd1a31 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.684865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1312/2621] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:39.686411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31532a-6400-4488-85f3-e290c263bb46 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.697268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31532a-6400-4488-85f3-e290c263bb46 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.702209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5142ad1-6e1a-4507-99ca-247002cbff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.702803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5142ad1-6e1a-4507-99ca-247002cbff24 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.703084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5142ad1-6e1a-4507-99ca-247002cbff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.703348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5142ad1-6e1a-4507-99ca-247002cbff24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.703981 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5142ad1-6e1a-4507-99ca-247002cbff24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.706103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5142ad1-6e1a-4507-99ca-247002cbff24 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.716780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1310/2622] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:39.721130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6afdc4-1e02-4545-9911-72cb3ffd1a31 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.731594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6afdc4-1e02-4545-9911-72cb3ffd1a31 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.736873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1c00f-fc6c-44c4-a5a5-afb242f0f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.737631 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1c00f-fc6c-44c4-a5a5-afb242f0f988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.738139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1c00f-fc6c-44c4-a5a5-afb242f0f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.738527 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1c00f-fc6c-44c4-a5a5-afb242f0f988 admin admin] PATH_INFO: `/v3/projects/bed6fd570c0a4eab8cb5fa8cd7dc6b2b/users/13083de6d7df44dd8c4b6be811f9331f/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.756007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1c00f-fc6c-44c4-a5a5-afb242f0f988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed6fd570c0a4eab8cb5fa8cd7dc6b2b, user_id=13083de6d7df44dd8c4b6be811f9331f, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.759006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1c00f-fc6c-44c4-a5a5-afb242f0f988 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.774077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1313/2623] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:39 2022] PUT /identity/v3/projects/bed6fd570c0a4eab8cb5fa8cd7dc6b2b/users/13083de6d7df44dd8c4b6be811f9331f/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:39.785371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796a11a-5dd5-48e9-9f0d-f6b69fca5eda None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.785738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796a11a-5dd5-48e9-9f0d-f6b69fca5eda None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.786148 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796a11a-5dd5-48e9-9f0d-f6b69fca5eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.817971 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6796a11a-5dd5-48e9-9f0d-f6b69fca5eda None None] MFA Rules not processed for user `13083de6d7df44dd8c4b6be811f9331f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:39.819259 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6796a11a-5dd5-48e9-9f0d-f6b69fca5eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:39.823537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:39.892160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1311/2624] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:39.896873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1c00f-fc6c-44c4-a5a5-afb242f0f988 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.974585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1c00f-fc6c-44c4-a5a5-afb242f0f988 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.979650 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55815dc4-a6c0-4453-a6db-fdd5274e81d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.980348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55815dc4-a6c0-4453-a6db-fdd5274e81d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.980630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55815dc4-a6c0-4453-a6db-fdd5274e81d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.981021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55815dc4-a6c0-4453-a6db-fdd5274e81d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.981695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55815dc4-a6c0-4453-a6db-fdd5274e81d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.983670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55815dc4-a6c0-4453-a6db-fdd5274e81d5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.026974 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1314/2625] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:39 2022] POST /identity/v3/projects => generated 380 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:40.028147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796a11a-5dd5-48e9-9f0d-f6b69fca5eda None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.046079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796a11a-5dd5-48e9-9f0d-f6b69fca5eda None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.113346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2647e6-1fac-47d5-94ca-4155b9cf8429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.114841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2647e6-1fac-47d5-94ca-4155b9cf8429 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.115113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2647e6-1fac-47d5-94ca-4155b9cf8429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.115454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2647e6-1fac-47d5-94ca-4155b9cf8429 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.116149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2647e6-1fac-47d5-94ca-4155b9cf8429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.118448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2647e6-1fac-47d5-94ca-4155b9cf8429 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.143198 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1312/2626] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:40 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:40.144410 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55815dc4-a6c0-4453-a6db-fdd5274e81d5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.155217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55815dc4-a6c0-4453-a6db-fdd5274e81d5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.160274 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f090812-cbc8-4d1b-99de-54fe76b36d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.161450 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f090812-cbc8-4d1b-99de-54fe76b36d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.161784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f090812-cbc8-4d1b-99de-54fe76b36d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.162057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f090812-cbc8-4d1b-99de-54fe76b36d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.162720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f090812-cbc8-4d1b-99de-54fe76b36d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.164307 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f090812-cbc8-4d1b-99de-54fe76b36d66 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.175901 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1315/2627] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:40 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:40.180845 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2647e6-1fac-47d5-94ca-4155b9cf8429 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.191064 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2647e6-1fac-47d5-94ca-4155b9cf8429 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.195952 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8e94c-eb69-4918-a667-7a19a81b0393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.196650 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8e94c-eb69-4918-a667-7a19a81b0393 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.196890 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8e94c-eb69-4918-a667-7a19a81b0393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.197158 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8e94c-eb69-4918-a667-7a19a81b0393 admin admin] PATH_INFO: `/v3/projects/b1fe1bb709084e22b0b2469ecba11ed3/users/4955db19a2d042ebaaa17ad888fc32ca/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.212286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8e94c-eb69-4918-a667-7a19a81b0393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1fe1bb709084e22b0b2469ecba11ed3, user_id=4955db19a2d042ebaaa17ad888fc32ca, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.214994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8e94c-eb69-4918-a667-7a19a81b0393 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.228905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1313/2628] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:40 2022] PUT /identity/v3/projects/b1fe1bb709084e22b0b2469ecba11ed3/users/4955db19a2d042ebaaa17ad888fc32ca/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:40.232967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f090812-cbc8-4d1b-99de-54fe76b36d66 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.244248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f090812-cbc8-4d1b-99de-54fe76b36d66 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.263677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:40.311024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2411d-ab96-487b-ab70-453daea34e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.311664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2411d-ab96-487b-ab70-453daea34e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.311949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2411d-ab96-487b-ab70-453daea34e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.312252 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2411d-ab96-487b-ab70-453daea34e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.312904 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2411d-ab96-487b-ab70-453daea34e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.314521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2411d-ab96-487b-ab70-453daea34e48 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.328189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1316/2629] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:40 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:40.329390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8e94c-eb69-4918-a667-7a19a81b0393 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.345118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8e94c-eb69-4918-a667-7a19a81b0393 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.352749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870373a-5707-4b31-b87d-fb9781bd376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.353440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870373a-5707-4b31-b87d-fb9781bd376e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.353863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870373a-5707-4b31-b87d-fb9781bd376e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.354141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870373a-5707-4b31-b87d-fb9781bd376e admin admin] PATH_INFO: `/v3/projects/b1fe1bb709084e22b0b2469ecba11ed3/users/4955db19a2d042ebaaa17ad888fc32ca/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.368048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870373a-5707-4b31-b87d-fb9781bd376e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1fe1bb709084e22b0b2469ecba11ed3, user_id=4955db19a2d042ebaaa17ad888fc32ca, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.371361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870373a-5707-4b31-b87d-fb9781bd376e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.397226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1314/2630] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:40 2022] PUT /identity/v3/projects/b1fe1bb709084e22b0b2469ecba11ed3/users/4955db19a2d042ebaaa17ad888fc32ca/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:40.398895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c70de-9cfb-403f-85e0-bc2fabf4b061 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.399185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c70de-9cfb-403f-85e0-bc2fabf4b061 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.399456 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c70de-9cfb-403f-85e0-bc2fabf4b061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.445201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-173c70de-9cfb-403f-85e0-bc2fabf4b061 None None] MFA Rules not processed for user `4955db19a2d042ebaaa17ad888fc32ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:40.448599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-173c70de-9cfb-403f-85e0-bc2fabf4b061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:40.540241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1317/2631] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:40.549865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870373a-5707-4b31-b87d-fb9781bd376e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.551762 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a870373a-5707-4b31-b87d-fb9781bd376e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:40.633948 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870373a-5707-4b31-b87d-fb9781bd376e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.638028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5991c-d75a-4618-a168-410f079cc90a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.638903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5991c-d75a-4618-a168-410f079cc90a service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.639211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5991c-d75a-4618-a168-410f079cc90a service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.639557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5991c-d75a-4618-a168-410f079cc90a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.652711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5991c-d75a-4618-a168-410f079cc90a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.654210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5991c-d75a-4618-a168-410f079cc90a service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.660408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:40.681075 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1315/2632] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:40 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:41.129448 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c70de-9cfb-403f-85e0-bc2fabf4b061 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.200239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c70de-9cfb-403f-85e0-bc2fabf4b061 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.205739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.206472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.206751 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.207021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] PATH_INFO: `/v3/users/13083de6d7df44dd8c4b6be811f9331f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.208713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13083de6d7df44dd8c4b6be811f9331f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.210896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.258568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5991c-d75a-4618-a168-410f079cc90a service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.260547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-79c5991c-d75a-4618-a168-410f079cc90a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:41.275931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5991c-d75a-4618-a168-410f079cc90a service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.280657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.281305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.281615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.282111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] PATH_INFO: `/v3/users/d3848643d2c44951b78814ccf7d504af` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.283320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3848643d2c44951b78814ccf7d504af)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.285313 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.340553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3848643d2c44951b78814ccf7d504af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.349479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3848643d2c44951b78814ccf7d504af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.350165 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3848643d2c44951b78814ccf7d504af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.362775 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3848643d2c44951b78814ccf7d504af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.371665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b4a756b0-8851-47b4-940e-35725fb63276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3848643d2c44951b78814ccf7d504af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.378295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1316/2633] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:41 2022] DELETE /identity/v3/users/d3848643d2c44951b78814ccf7d504af => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:41.390048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:41.391818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13083de6d7df44dd8c4b6be811f9331f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.406970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13083de6d7df44dd8c4b6be811f9331f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.415139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13083de6d7df44dd8c4b6be811f9331f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.423949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13083de6d7df44dd8c4b6be811f9331f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.424799 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13083de6d7df44dd8c4b6be811f9331f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.431894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1318/2634] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:41 2022] DELETE /identity/v3/users/13083de6d7df44dd8c4b6be811f9331f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:41.436296 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc256d5-07b7-4aa3-ad70-2f93af3bd608 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.436660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc256d5-07b7-4aa3-ad70-2f93af3bd608 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.437002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc256d5-07b7-4aa3-ad70-2f93af3bd608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.457402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7cc256d5-07b7-4aa3-ad70-2f93af3bd608 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:41.459864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7cc256d5-07b7-4aa3-ad70-2f93af3bd608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:41.524270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1317/2635] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:41.534183 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.536293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:41.570773 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc256d5-07b7-4aa3-ad70-2f93af3bd608 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.572352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7cc256d5-07b7-4aa3-ad70-2f93af3bd608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:41.585121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc256d5-07b7-4aa3-ad70-2f93af3bd608 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.589906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.590545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.590825 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.591098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] PATH_INFO: `/v3/projects/1cf7216d5c904b24a9665755720cf287` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.592243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf7216d5c904b24a9665755720cf287)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.594197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.628126 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f648082-368c-4be4-bf85-11ef47a7e558 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.644213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf7216d5c904b24a9665755720cf287', 'request_id': 'req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.652607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf7216d5c904b24a9665755720cf287', 'request_id': 'req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.658918 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf7216d5c904b24a9665755720cf287', 'request_id': 'req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.662818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:41.674520 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] The token cache is being invalidate because project 1cf7216d5c904b24a9665755720cf287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:41.675167 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.676039 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b044e3d-bd6b-4a72-845b-eeccf723e5a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:41.678393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1318/2636] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:41 2022] DELETE /identity/v3/projects/1cf7216d5c904b24a9665755720cf287 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:41.683282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250698-6aa3-450e-a4cb-487561e5156e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.683723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250698-6aa3-450e-a4cb-487561e5156e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.684004 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250698-6aa3-450e-a4cb-487561e5156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.688503 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f544d-a209-4835-9299-5cc2ddd3eefe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.689244 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f544d-a209-4835-9299-5cc2ddd3eefe service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.689534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f544d-a209-4835-9299-5cc2ddd3eefe service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.689843 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f544d-a209-4835-9299-5cc2ddd3eefe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.691781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-242f544d-a209-4835-9299-5cc2ddd3eefe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:41.704457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b7250698-6aa3-450e-a4cb-487561e5156e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:41.705677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b7250698-6aa3-450e-a4cb-487561e5156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:41.756287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f544d-a209-4835-9299-5cc2ddd3eefe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.757756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f544d-a209-4835-9299-5cc2ddd3eefe service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.766818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1319/2637] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:44:41.767446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1319/2638] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:41.776647 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f839110-4541-4456-9eef-3a7480733736 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.776932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f839110-4541-4456-9eef-3a7480733736 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.777228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f839110-4541-4456-9eef-3a7480733736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.797761 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9f839110-4541-4456-9eef-3a7480733736 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:41.798979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9f839110-4541-4456-9eef-3a7480733736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:41.807947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1320/2639] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:44:41.812420 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f544d-a209-4835-9299-5cc2ddd3eefe service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.823269 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f544d-a209-4835-9299-5cc2ddd3eefe service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.828618 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712304f-83dd-457a-82ec-bd96c42c16a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.829339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712304f-83dd-457a-82ec-bd96c42c16a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.829767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712304f-83dd-457a-82ec-bd96c42c16a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.830139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712304f-83dd-457a-82ec-bd96c42c16a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.830917 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712304f-83dd-457a-82ec-bd96c42c16a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:41.831295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712304f-83dd-457a-82ec-bd96c42c16a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.833589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712304f-83dd-457a-82ec-bd96c42c16a0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.847120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1320/2640] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:41.852036 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f839110-4541-4456-9eef-3a7480733736 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.862398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f839110-4541-4456-9eef-3a7480733736 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.867558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57a6e9-21bd-40ec-aa16-65c0df533886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.868163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57a6e9-21bd-40ec-aa16-65c0df533886 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.868447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57a6e9-21bd-40ec-aa16-65c0df533886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.868720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57a6e9-21bd-40ec-aa16-65c0df533886 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.869442 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57a6e9-21bd-40ec-aa16-65c0df533886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.871826 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57a6e9-21bd-40ec-aa16-65c0df533886 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.902038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1321/2641] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:41 2022] POST /identity/v3/projects => generated 368 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:41.906081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712304f-83dd-457a-82ec-bd96c42c16a0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.917210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712304f-83dd-457a-82ec-bd96c42c16a0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.932778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:41.983440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8129c-be88-4f9c-afe3-e2d2c5a659f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.984052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8129c-be88-4f9c-afe3-e2d2c5a659f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.984329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8129c-be88-4f9c-afe3-e2d2c5a659f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.984600 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8129c-be88-4f9c-afe3-e2d2c5a659f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.985228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8129c-be88-4f9c-afe3-e2d2c5a659f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.987528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8129c-be88-4f9c-afe3-e2d2c5a659f9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.011557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1321/2642] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:41 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:42.015670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57a6e9-21bd-40ec-aa16-65c0df533886 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.026415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57a6e9-21bd-40ec-aa16-65c0df533886 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.031852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0331a5-c9a4-45cb-97d1-fbe4f66f27a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.032555 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0331a5-c9a4-45cb-97d1-fbe4f66f27a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.032855 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0331a5-c9a4-45cb-97d1-fbe4f66f27a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.033124 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0331a5-c9a4-45cb-97d1-fbe4f66f27a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.033858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0331a5-c9a4-45cb-97d1-fbe4f66f27a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.035457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0331a5-c9a4-45cb-97d1-fbe4f66f27a1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.046978 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1322/2643] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:42 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:42.051722 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8129c-be88-4f9c-afe3-e2d2c5a659f9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.059934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0331a5-c9a4-45cb-97d1-fbe4f66f27a1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.062046 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4a0331a5-c9a4-45cb-97d1-fbe4f66f27a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:42.062530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8129c-be88-4f9c-afe3-e2d2c5a659f9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.067776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569f886-871b-4262-a374-8ae0508d7404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.068698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569f886-871b-4262-a374-8ae0508d7404 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.068976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569f886-871b-4262-a374-8ae0508d7404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.069247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569f886-871b-4262-a374-8ae0508d7404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.070030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569f886-871b-4262-a374-8ae0508d7404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.071724 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569f886-871b-4262-a374-8ae0508d7404 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.078894 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0331a5-c9a4-45cb-97d1-fbe4f66f27a1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.082920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1322/2644] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:42 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:42.087379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569f886-871b-4262-a374-8ae0508d7404 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.087782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.088475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.088751 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.089022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] PATH_INFO: `/v3/projects/bed6fd570c0a4eab8cb5fa8cd7dc6b2b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.090176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed6fd570c0a4eab8cb5fa8cd7dc6b2b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.092079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.097802 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569f886-871b-4262-a374-8ae0508d7404 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.102594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794327e2-14e6-42d4-b93b-6168367002bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.103250 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794327e2-14e6-42d4-b93b-6168367002bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.103530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794327e2-14e6-42d4-b93b-6168367002bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.103831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794327e2-14e6-42d4-b93b-6168367002bc admin admin] PATH_INFO: `/v3/projects/dc5815cf211c428e92f7e51020d24b7f/users/c846a21be84f439abb660b017087f6d7/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.123172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794327e2-14e6-42d4-b93b-6168367002bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5815cf211c428e92f7e51020d24b7f, user_id=c846a21be84f439abb660b017087f6d7, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.126014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794327e2-14e6-42d4-b93b-6168367002bc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.140731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1323/2645] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:42 2022] PUT /identity/v3/projects/dc5815cf211c428e92f7e51020d24b7f/users/c846a21be84f439abb660b017087f6d7/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:42.143135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed6fd570c0a4eab8cb5fa8cd7dc6b2b', 'request_id': 'req-edeff8ba-a964-4039-b617-d9b909df0f1c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.150308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed6fd570c0a4eab8cb5fa8cd7dc6b2b', 'request_id': 'req-edeff8ba-a964-4039-b617-d9b909df0f1c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.151818 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca0975-f81d-4bd9-9def-994cde3594a6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.152290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca0975-f81d-4bd9-9def-994cde3594a6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.152705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca0975-f81d-4bd9-9def-994cde3594a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.156568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed6fd570c0a4eab8cb5fa8cd7dc6b2b', 'request_id': 'req-edeff8ba-a964-4039-b617-d9b909df0f1c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.171829 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] The token cache is being invalidate because project bed6fd570c0a4eab8cb5fa8cd7dc6b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:42.172118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.172964 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.174584 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1323/2646] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:42 2022] DELETE /identity/v3/projects/bed6fd570c0a4eab8cb5fa8cd7dc6b2b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:42.179407 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.180844 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:42.187627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-25ca0975-f81d-4bd9-9def-994cde3594a6 None None] MFA Rules not processed for user `c846a21be84f439abb660b017087f6d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:42.188816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-25ca0975-f81d-4bd9-9def-994cde3594a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:42.223863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:42.248024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeff8ba-a964-4039-b617-d9b909df0f1c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.261516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1324/2647] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:44:42.269686 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca0975-f81d-4bd9-9def-994cde3594a6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.271416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-25ca0975-f81d-4bd9-9def-994cde3594a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:42.321055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.321867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.322195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.322537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] PATH_INFO: `/v3/users/4955db19a2d042ebaaa17ad888fc32ca` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.323774 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4955db19a2d042ebaaa17ad888fc32ca)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.326112 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.356108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca0975-f81d-4bd9-9def-994cde3594a6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.360309 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0424b-6613-4028-9c7f-eed52323f656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.360940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0424b-6613-4028-9c7f-eed52323f656 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.361240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0424b-6613-4028-9c7f-eed52323f656 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.361860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0424b-6613-4028-9c7f-eed52323f656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.372524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0424b-6613-4028-9c7f-eed52323f656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.373994 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0424b-6613-4028-9c7f-eed52323f656 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.379869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4955db19a2d042ebaaa17ad888fc32ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.383559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1325/2648] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:42.387391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4955db19a2d042ebaaa17ad888fc32ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.388133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4955db19a2d042ebaaa17ad888fc32ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.402624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4955db19a2d042ebaaa17ad888fc32ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.410261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c28a0937-0dc6-4396-b6b6-01c5f4ce7e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4955db19a2d042ebaaa17ad888fc32ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.417192 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1324/2649] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:42 2022] DELETE /identity/v3/users/4955db19a2d042ebaaa17ad888fc32ca => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:42.730990 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0424b-6613-4028-9c7f-eed52323f656 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.737254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:42.747671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0424b-6613-4028-9c7f-eed52323f656 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.811727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.812350 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.812626 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.812898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] PATH_INFO: `/v3/projects/b1fe1bb709084e22b0b2469ecba11ed3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.814091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1fe1bb709084e22b0b2469ecba11ed3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.816127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.863522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1fe1bb709084e22b0b2469ecba11ed3', 'request_id': 'req-685ff16f-4193-45ed-8bc7-5840b6efb715'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.871740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1fe1bb709084e22b0b2469ecba11ed3', 'request_id': 'req-685ff16f-4193-45ed-8bc7-5840b6efb715'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.877305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1fe1bb709084e22b0b2469ecba11ed3', 'request_id': 'req-685ff16f-4193-45ed-8bc7-5840b6efb715'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.889015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] The token cache is being invalidate because project b1fe1bb709084e22b0b2469ecba11ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:42.889288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.889948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-685ff16f-4193-45ed-8bc7-5840b6efb715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:42.891416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1326/2650] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:42 2022] DELETE /identity/v3/projects/b1fe1bb709084e22b0b2469ecba11ed3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:42.896297 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28542a5-d807-4d45-9ad7-04bbfaccfe45 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.896586 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28542a5-d807-4d45-9ad7-04bbfaccfe45 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.896861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28542a5-d807-4d45-9ad7-04bbfaccfe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.918146 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a28542a5-d807-4d45-9ad7-04bbfaccfe45 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:42.919545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a28542a5-d807-4d45-9ad7-04bbfaccfe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:43.106052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1325/2651] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:43.114841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4700b-e871-4d70-a69c-ac82977c20ab None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.115068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4700b-e871-4d70-a69c-ac82977c20ab None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.115341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4700b-e871-4d70-a69c-ac82977c20ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.135681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-07c4700b-e871-4d70-a69c-ac82977c20ab None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:43.136893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-07c4700b-e871-4d70-a69c-ac82977c20ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:43.144701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1327/2652] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:43.148974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28542a5-d807-4d45-9ad7-04bbfaccfe45 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.158550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28542a5-d807-4d45-9ad7-04bbfaccfe45 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.163675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4338067-6dc6-43f8-8856-9a5802fa0057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.164262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4338067-6dc6-43f8-8856-9a5802fa0057 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.164534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4338067-6dc6-43f8-8856-9a5802fa0057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.164799 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4338067-6dc6-43f8-8856-9a5802fa0057 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.165432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4338067-6dc6-43f8-8856-9a5802fa0057 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:43.166026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4338067-6dc6-43f8-8856-9a5802fa0057 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.167586 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4338067-6dc6-43f8-8856-9a5802fa0057 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.169043 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:43.181683 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1326/2653] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:43.186105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4700b-e871-4d70-a69c-ac82977c20ab None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.195278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4700b-e871-4d70-a69c-ac82977c20ab None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.200015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80f221-92d9-49df-8e2f-a076c25a7f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.200599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80f221-92d9-49df-8e2f-a076c25a7f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.200879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80f221-92d9-49df-8e2f-a076c25a7f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.201151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80f221-92d9-49df-8e2f-a076c25a7f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.201823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80f221-92d9-49df-8e2f-a076c25a7f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.203721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80f221-92d9-49df-8e2f-a076c25a7f26 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.231087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1328/2654] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:43 2022] POST /identity/v3/projects => generated 374 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:43.235261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4338067-6dc6-43f8-8856-9a5802fa0057 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.246395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4338067-6dc6-43f8-8856-9a5802fa0057 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.304113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb90ad9-ea9d-4fa3-b3bd-b4bde3942c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.304724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb90ad9-ea9d-4fa3-b3bd-b4bde3942c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.305002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb90ad9-ea9d-4fa3-b3bd-b4bde3942c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.305271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb90ad9-ea9d-4fa3-b3bd-b4bde3942c09 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.305945 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb90ad9-ea9d-4fa3-b3bd-b4bde3942c09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.307859 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb90ad9-ea9d-4fa3-b3bd-b4bde3942c09 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.328438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1327/2655] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:43 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:43.332653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80f221-92d9-49df-8e2f-a076c25a7f26 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.343024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80f221-92d9-49df-8e2f-a076c25a7f26 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.347534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37d813-7159-48a4-aec1-28b745ec41a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.348621 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37d813-7159-48a4-aec1-28b745ec41a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.349041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37d813-7159-48a4-aec1-28b745ec41a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.349450 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37d813-7159-48a4-aec1-28b745ec41a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.350121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37d813-7159-48a4-aec1-28b745ec41a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.351670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37d813-7159-48a4-aec1-28b745ec41a7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.361295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1329/2656] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:43.366072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb90ad9-ea9d-4fa3-b3bd-b4bde3942c09 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.377596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb90ad9-ea9d-4fa3-b3bd-b4bde3942c09 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.382570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0171f-0ced-487a-a048-eca6ef5e24df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.383172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0171f-0ced-487a-a048-eca6ef5e24df admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.383453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0171f-0ced-487a-a048-eca6ef5e24df admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.383728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0171f-0ced-487a-a048-eca6ef5e24df admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.384364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0171f-0ced-487a-a048-eca6ef5e24df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.386051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0171f-0ced-487a-a048-eca6ef5e24df admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.396490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1328/2657] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:43.400718 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37d813-7159-48a4-aec1-28b745ec41a7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.410362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37d813-7159-48a4-aec1-28b745ec41a7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.415952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fb9cc-f725-472c-9b49-9cb217b69df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.416588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fb9cc-f725-472c-9b49-9cb217b69df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.416861 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fb9cc-f725-472c-9b49-9cb217b69df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.417127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fb9cc-f725-472c-9b49-9cb217b69df6 admin admin] PATH_INFO: `/v3/projects/fb94475797a3463093c30f61eff2193a/users/8afd7b912a4f45af80765383201f1c70/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.431951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fb9cc-f725-472c-9b49-9cb217b69df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb94475797a3463093c30f61eff2193a, user_id=8afd7b912a4f45af80765383201f1c70, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.434582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fb9cc-f725-472c-9b49-9cb217b69df6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.447257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1330/2658] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:43 2022] PUT /identity/v3/projects/fb94475797a3463093c30f61eff2193a/users/8afd7b912a4f45af80765383201f1c70/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:43.452201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972a58c-6c73-481a-ab13-5eb3bc27ea6c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.452487 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972a58c-6c73-481a-ab13-5eb3bc27ea6c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.452753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972a58c-6c73-481a-ab13-5eb3bc27ea6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.472267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0972a58c-6c73-481a-ab13-5eb3bc27ea6c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:43.474352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0972a58c-6c73-481a-ab13-5eb3bc27ea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:43.533972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1329/2659] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:43.544769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fb9cc-f725-472c-9b49-9cb217b69df6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.546507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9f4fb9cc-f725-472c-9b49-9cb217b69df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:43.578142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:43.605952 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fb9cc-f725-472c-9b49-9cb217b69df6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.609674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9de6ed-e37d-47e1-b813-14386702baad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.610276 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9de6ed-e37d-47e1-b813-14386702baad service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.610552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9de6ed-e37d-47e1-b813-14386702baad service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.610820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9de6ed-e37d-47e1-b813-14386702baad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.620564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9de6ed-e37d-47e1-b813-14386702baad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.621945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9de6ed-e37d-47e1-b813-14386702baad service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.631868 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1331/2660] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:43.820458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972a58c-6c73-481a-ab13-5eb3bc27ea6c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.822247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0972a58c-6c73-481a-ab13-5eb3bc27ea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:43.925273 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972a58c-6c73-481a-ab13-5eb3bc27ea6c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.935045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e65468-5d05-4d37-a58f-d40fd4d1d027 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.935872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e65468-5d05-4d37-a58f-d40fd4d1d027 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.936499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e65468-5d05-4d37-a58f-d40fd4d1d027 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.936915 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e65468-5d05-4d37-a58f-d40fd4d1d027 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.939662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f5e65468-5d05-4d37-a58f-d40fd4d1d027 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:43.955527 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e65468-5d05-4d37-a58f-d40fd4d1d027 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.957323 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e65468-5d05-4d37-a58f-d40fd4d1d027 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.972764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1330/2661] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:44.408347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2815e-5276-42a2-be28-a0dae9cb6498 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.408831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2815e-5276-42a2-be28-a0dae9cb6498 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.408949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2815e-5276-42a2-be28-a0dae9cb6498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.438109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-24c2815e-5276-42a2-be28-a0dae9cb6498 None None] MFA Rules not processed for user `8afd7b912a4f45af80765383201f1c70`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:44.439329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-24c2815e-5276-42a2-be28-a0dae9cb6498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:44.514300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1332/2662] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:44:44.515441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e65468-5d05-4d37-a58f-d40fd4d1d027 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.521783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:44.528486 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e65468-5d05-4d37-a58f-d40fd4d1d027 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.534195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e809-a2f5-4b2b-9d92-2b51fe995da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.534833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e809-a2f5-4b2b-9d92-2b51fe995da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.535117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e809-a2f5-4b2b-9d92-2b51fe995da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.535388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e809-a2f5-4b2b-9d92-2b51fe995da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.536705 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e809-a2f5-4b2b-9d92-2b51fe995da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.538720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e809-a2f5-4b2b-9d92-2b51fe995da0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.572846 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1331/2663] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:44 2022] POST /identity/v3/projects => generated 374 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:44.577007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2815e-5276-42a2-be28-a0dae9cb6498 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.588654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2815e-5276-42a2-be28-a0dae9cb6498 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.652184 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aeadc-2140-4716-a8c4-64c926ef6188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.652809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aeadc-2140-4716-a8c4-64c926ef6188 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.653091 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aeadc-2140-4716-a8c4-64c926ef6188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.653366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aeadc-2140-4716-a8c4-64c926ef6188 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.654037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aeadc-2140-4716-a8c4-64c926ef6188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.655951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aeadc-2140-4716-a8c4-64c926ef6188 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.680222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1333/2664] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:44 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:44.684418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e809-a2f5-4b2b-9d92-2b51fe995da0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.694916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e809-a2f5-4b2b-9d92-2b51fe995da0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.699542 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7d6f-f87b-44ac-94c8-a542e943b64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.700144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7d6f-f87b-44ac-94c8-a542e943b64c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.700418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7d6f-f87b-44ac-94c8-a542e943b64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.700682 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7d6f-f87b-44ac-94c8-a542e943b64c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.701312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7d6f-f87b-44ac-94c8-a542e943b64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.702908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7d6f-f87b-44ac-94c8-a542e943b64c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.714012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1332/2665] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:44 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:44.718367 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aeadc-2140-4716-a8c4-64c926ef6188 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.729213 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aeadc-2140-4716-a8c4-64c926ef6188 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.733822 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe683351-8ffd-405b-9c1e-9e7f5220d7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.734472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe683351-8ffd-405b-9c1e-9e7f5220d7aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.734747 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe683351-8ffd-405b-9c1e-9e7f5220d7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.735012 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe683351-8ffd-405b-9c1e-9e7f5220d7aa admin admin] PATH_INFO: `/v3/projects/c1b2bf6fa8bd4787a32659e335949dc3/users/ce46630863374be6895ec73eebfba9cf/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.750510 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe683351-8ffd-405b-9c1e-9e7f5220d7aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b2bf6fa8bd4787a32659e335949dc3, user_id=ce46630863374be6895ec73eebfba9cf, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.753109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe683351-8ffd-405b-9c1e-9e7f5220d7aa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.765766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1334/2666] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:44 2022] PUT /identity/v3/projects/c1b2bf6fa8bd4787a32659e335949dc3/users/ce46630863374be6895ec73eebfba9cf/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:44.769809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7d6f-f87b-44ac-94c8-a542e943b64c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.780458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7d6f-f87b-44ac-94c8-a542e943b64c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.841318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259802ac-fcde-4f9a-b43a-05192a22691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.841959 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259802ac-fcde-4f9a-b43a-05192a22691a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.842239 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259802ac-fcde-4f9a-b43a-05192a22691a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.842509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259802ac-fcde-4f9a-b43a-05192a22691a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.843150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259802ac-fcde-4f9a-b43a-05192a22691a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.844817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259802ac-fcde-4f9a-b43a-05192a22691a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.855812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1333/2667] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:44 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:44.860131 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe683351-8ffd-405b-9c1e-9e7f5220d7aa admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.871382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe683351-8ffd-405b-9c1e-9e7f5220d7aa admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.876109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f9e21-aa12-4f1d-ae34-c1e6902ac8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.876782 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f9e21-aa12-4f1d-ae34-c1e6902ac8f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.877056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f9e21-aa12-4f1d-ae34-c1e6902ac8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.877326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f9e21-aa12-4f1d-ae34-c1e6902ac8f2 admin admin] PATH_INFO: `/v3/projects/c1b2bf6fa8bd4787a32659e335949dc3/users/ce46630863374be6895ec73eebfba9cf/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.880271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f9e21-aa12-4f1d-ae34-c1e6902ac8f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b2bf6fa8bd4787a32659e335949dc3, user_id=ce46630863374be6895ec73eebfba9cf, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.883185 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f9e21-aa12-4f1d-ae34-c1e6902ac8f2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.895648 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1335/2668] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:44 2022] PUT /identity/v3/projects/c1b2bf6fa8bd4787a32659e335949dc3/users/ce46630863374be6895ec73eebfba9cf/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:45.648482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c74975-55d2-41f3-9b6d-1e6d8daf1340 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.648987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c74975-55d2-41f3-9b6d-1e6d8daf1340 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.649665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c74975-55d2-41f3-9b6d-1e6d8daf1340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.718912 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-12c74975-55d2-41f3-9b6d-1e6d8daf1340 None None] MFA Rules not processed for user `ce46630863374be6895ec73eebfba9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:45.720217 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-12c74975-55d2-41f3-9b6d-1e6d8daf1340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:45.814081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1334/2669] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:45.826366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f9e21-aa12-4f1d-ae34-c1e6902ac8f2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:45.828141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b67f9e21-aa12-4f1d-ae34-c1e6902ac8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:45.847933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:45.890553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f9e21-aa12-4f1d-ae34-c1e6902ac8f2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:45.894186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8217c2-ab7c-479d-b63e-65360080c2fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:45.894791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8217c2-ab7c-479d-b63e-65360080c2fa service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.895602 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8217c2-ab7c-479d-b63e-65360080c2fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.895873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8217c2-ab7c-479d-b63e-65360080c2fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.952647 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8217c2-ab7c-479d-b63e-65360080c2fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:45.954110 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8217c2-ab7c-479d-b63e-65360080c2fa service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:45.963492 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1336/2670] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:47.326725 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c74975-55d2-41f3-9b6d-1e6d8daf1340 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:47.336874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c74975-55d2-41f3-9b6d-1e6d8daf1340 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:47.340889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986fea5-e785-49e4-831e-14907eb65a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:47.341166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986fea5-e785-49e4-831e-14907eb65a2a service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:47.341440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986fea5-e785-49e4-831e-14907eb65a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:47.341738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986fea5-e785-49e4-831e-14907eb65a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:47.351572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986fea5-e785-49e4-831e-14907eb65a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:47.352995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986fea5-e785-49e4-831e-14907eb65a2a service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:47.362989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1335/2671] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:51.153335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8217c2-ab7c-479d-b63e-65360080c2fa service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:51.154803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8b8217c2-ab7c-479d-b63e-65360080c2fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:51.219961 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8217c2-ab7c-479d-b63e-65360080c2fa service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:51.225506 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:51.225971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:51.226422 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:51.226609 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] PATH_INFO: `/v3/users/c846a21be84f439abb660b017087f6d7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:51.227610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c846a21be84f439abb660b017087f6d7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:51.229520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:51.279935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c846a21be84f439abb660b017087f6d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:51.293860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c846a21be84f439abb660b017087f6d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:51.298369 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:51.301514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c846a21be84f439abb660b017087f6d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:51.310440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c846a21be84f439abb660b017087f6d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:51.311132 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c846a21be84f439abb660b017087f6d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:51.317767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1337/2672] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:51 2022] DELETE /identity/v3/users/c846a21be84f439abb660b017087f6d7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:51.322291 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b6866-ba2d-43db-b394-84bf4bf342ad None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:51.322801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b6866-ba2d-43db-b394-84bf4bf342ad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:51.322891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b6866-ba2d-43db-b394-84bf4bf342ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:51.345045 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2f1b6866-ba2d-43db-b394-84bf4bf342ad None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:51.346324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2f1b6866-ba2d-43db-b394-84bf4bf342ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:51.384280 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:51.416172 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1336/2673] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:51.428752 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:51.430494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:51.543077 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abdc51-bebe-4945-8048-26f519620288 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:51.548364 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3874aca-b943-4f07-9a2a-95256fc95aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:51.549179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3874aca-b943-4f07-9a2a-95256fc95aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:51.549554 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3874aca-b943-4f07-9a2a-95256fc95aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:51.549865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3874aca-b943-4f07-9a2a-95256fc95aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:51.563968 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3874aca-b943-4f07-9a2a-95256fc95aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:51.565114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3874aca-b943-4f07-9a2a-95256fc95aa7 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:51.576817 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1338/2674] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:44:51.903169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b6866-ba2d-43db-b394-84bf4bf342ad None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:51.904903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2f1b6866-ba2d-43db-b394-84bf4bf342ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:51.919698 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b6866-ba2d-43db-b394-84bf4bf342ad None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:51.924298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:51.925496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:51.925808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:51.926792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] PATH_INFO: `/v3/projects/dc5815cf211c428e92f7e51020d24b7f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:51.927932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5815cf211c428e92f7e51020d24b7f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:51.929865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:51.976338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5815cf211c428e92f7e51020d24b7f', 'request_id': 'req-0598be25-9930-4b71-8970-cf9071f718ef'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:51.982308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5815cf211c428e92f7e51020d24b7f', 'request_id': 'req-0598be25-9930-4b71-8970-cf9071f718ef'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:51.988079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5815cf211c428e92f7e51020d24b7f', 'request_id': 'req-0598be25-9930-4b71-8970-cf9071f718ef'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:52.000948 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] The token cache is being invalidate because project dc5815cf211c428e92f7e51020d24b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:52.001231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:52.001874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0598be25-9930-4b71-8970-cf9071f718ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:52.003321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1337/2675] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:51 2022] DELETE /identity/v3/projects/dc5815cf211c428e92f7e51020d24b7f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:52.009089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788680f-20d1-4cfc-ac0c-d32bbfeceb1c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.009376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788680f-20d1-4cfc-ac0c-d32bbfeceb1c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.009688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788680f-20d1-4cfc-ac0c-d32bbfeceb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.029265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6788680f-20d1-4cfc-ac0c-d32bbfeceb1c None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:52.030502 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6788680f-20d1-4cfc-ac0c-d32bbfeceb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:52.089712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1339/2676] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:52.097477 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fceb93-54f7-45b6-8c53-eed8c7cf161a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.098633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fceb93-54f7-45b6-8c53-eed8c7cf161a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.098815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fceb93-54f7-45b6-8c53-eed8c7cf161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.119149 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-22fceb93-54f7-45b6-8c53-eed8c7cf161a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:52.120476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-22fceb93-54f7-45b6-8c53-eed8c7cf161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:52.128360 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1338/2677] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:52.132954 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788680f-20d1-4cfc-ac0c-d32bbfeceb1c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.145075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788680f-20d1-4cfc-ac0c-d32bbfeceb1c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.149617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593067f9-ae06-48cc-94e2-3da1cfd3ddb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.150148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593067f9-ae06-48cc-94e2-3da1cfd3ddb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.150426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593067f9-ae06-48cc-94e2-3da1cfd3ddb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.150821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593067f9-ae06-48cc-94e2-3da1cfd3ddb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.151464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593067f9-ae06-48cc-94e2-3da1cfd3ddb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:52.151796 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593067f9-ae06-48cc-94e2-3da1cfd3ddb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.153367 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593067f9-ae06-48cc-94e2-3da1cfd3ddb2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.165642 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1340/2678] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:52.170343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fceb93-54f7-45b6-8c53-eed8c7cf161a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.180318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fceb93-54f7-45b6-8c53-eed8c7cf161a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.185363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4901080-ccf8-4694-a46a-99cbd3c30d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.185990 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4901080-ccf8-4694-a46a-99cbd3c30d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.186270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4901080-ccf8-4694-a46a-99cbd3c30d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.186540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4901080-ccf8-4694-a46a-99cbd3c30d7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.187176 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4901080-ccf8-4694-a46a-99cbd3c30d7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.189061 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4901080-ccf8-4694-a46a-99cbd3c30d7f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.215764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1339/2679] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:52 2022] POST /identity/v3/projects => generated 394 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:52.220147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593067f9-ae06-48cc-94e2-3da1cfd3ddb2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.230665 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593067f9-ae06-48cc-94e2-3da1cfd3ddb2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.280637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:52.287674 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47ca3e-0009-4aeb-af62-95bcb4d25d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.288466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ca3e-0009-4aeb-af62-95bcb4d25d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.288987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ca3e-0009-4aeb-af62-95bcb4d25d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.289658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ca3e-0009-4aeb-af62-95bcb4d25d53 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.290462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47ca3e-0009-4aeb-af62-95bcb4d25d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.292816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47ca3e-0009-4aeb-af62-95bcb4d25d53 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.315645 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1341/2680] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:52 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:52.320160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4901080-ccf8-4694-a46a-99cbd3c30d7f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.330289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4901080-ccf8-4694-a46a-99cbd3c30d7f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.335316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7f58c-4307-41c0-a239-c19f3c9ad98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.335724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7f58c-4307-41c0-a239-c19f3c9ad98a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.335997 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7f58c-4307-41c0-a239-c19f3c9ad98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.336261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7f58c-4307-41c0-a239-c19f3c9ad98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.336891 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7f58c-4307-41c0-a239-c19f3c9ad98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.338497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7f58c-4307-41c0-a239-c19f3c9ad98a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.348723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1340/2681] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:52.353129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47ca3e-0009-4aeb-af62-95bcb4d25d53 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.363019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47ca3e-0009-4aeb-af62-95bcb4d25d53 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.368457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68144f-a88c-4ded-bd07-cdd5f16fb123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.369048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68144f-a88c-4ded-bd07-cdd5f16fb123 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.369326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68144f-a88c-4ded-bd07-cdd5f16fb123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.369622 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68144f-a88c-4ded-bd07-cdd5f16fb123 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.370257 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68144f-a88c-4ded-bd07-cdd5f16fb123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.371833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68144f-a88c-4ded-bd07-cdd5f16fb123 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.382806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1342/2682] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:52.387207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7f58c-4307-41c0-a239-c19f3c9ad98a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.398424 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7f58c-4307-41c0-a239-c19f3c9ad98a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.403121 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2ab37-7aef-4c9b-8423-66b2525e9062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.403717 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2ab37-7aef-4c9b-8423-66b2525e9062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.403998 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2ab37-7aef-4c9b-8423-66b2525e9062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.404271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2ab37-7aef-4c9b-8423-66b2525e9062 admin admin] PATH_INFO: `/v3/projects/d9afbc1494cb467299917d0c17592840/users/2d7a005fea2f4a83826da1fcbb64f4a6/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.418901 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2ab37-7aef-4c9b-8423-66b2525e9062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9afbc1494cb467299917d0c17592840, user_id=2d7a005fea2f4a83826da1fcbb64f4a6, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.421567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2ab37-7aef-4c9b-8423-66b2525e9062 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.437722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1341/2683] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:52 2022] PUT /identity/v3/projects/d9afbc1494cb467299917d0c17592840/users/2d7a005fea2f4a83826da1fcbb64f4a6/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:52.450109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff9a47-667b-4bbf-abf7-00c5328bf87d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.450396 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff9a47-667b-4bbf-abf7-00c5328bf87d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.450671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff9a47-667b-4bbf-abf7-00c5328bf87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.482926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ccff9a47-667b-4bbf-abf7-00c5328bf87d None None] MFA Rules not processed for user `2d7a005fea2f4a83826da1fcbb64f4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:52.484137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ccff9a47-667b-4bbf-abf7-00c5328bf87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:52.560437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1343/2684] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:52.565215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2ab37-7aef-4c9b-8423-66b2525e9062 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.637139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:52.641378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2ab37-7aef-4c9b-8423-66b2525e9062 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.652311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a86ba-208d-43bc-8b96-5aafb53c0de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.652936 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a86ba-208d-43bc-8b96-5aafb53c0de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.653282 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a86ba-208d-43bc-8b96-5aafb53c0de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.653498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a86ba-208d-43bc-8b96-5aafb53c0de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.654147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a86ba-208d-43bc-8b96-5aafb53c0de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.656024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a86ba-208d-43bc-8b96-5aafb53c0de0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.688131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1342/2685] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:52 2022] POST /identity/v3/projects => generated 392 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:52.692364 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff9a47-667b-4bbf-abf7-00c5328bf87d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.705894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff9a47-667b-4bbf-abf7-00c5328bf87d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.775310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2104f-be58-4ab6-9f86-5882270e5772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.776144 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2104f-be58-4ab6-9f86-5882270e5772 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.776546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2104f-be58-4ab6-9f86-5882270e5772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.776894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2104f-be58-4ab6-9f86-5882270e5772 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.777693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2104f-be58-4ab6-9f86-5882270e5772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.779807 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2104f-be58-4ab6-9f86-5882270e5772 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.809854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1344/2686] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:52 2022] POST /identity/v3/users => generated 303 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:52.810745 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a86ba-208d-43bc-8b96-5aafb53c0de0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.823710 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a86ba-208d-43bc-8b96-5aafb53c0de0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.837262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03985-ef7a-4761-8557-45d12e73833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.838378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03985-ef7a-4761-8557-45d12e73833c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.838729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03985-ef7a-4761-8557-45d12e73833c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.839071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03985-ef7a-4761-8557-45d12e73833c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.839800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a03985-ef7a-4761-8557-45d12e73833c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.842347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a03985-ef7a-4761-8557-45d12e73833c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.856926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1343/2687] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:52.860777 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2104f-be58-4ab6-9f86-5882270e5772 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.872039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2104f-be58-4ab6-9f86-5882270e5772 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.885955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da02f7-da89-4829-9112-fc5028d0d474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.886766 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da02f7-da89-4829-9112-fc5028d0d474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.887048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da02f7-da89-4829-9112-fc5028d0d474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.887325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da02f7-da89-4829-9112-fc5028d0d474 admin admin] PATH_INFO: `/v3/projects/7f0d02a748d54e5e97f80d163db5cbba/users/be877ff3bb304e589eaffa96709882f6/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.903959 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da02f7-da89-4829-9112-fc5028d0d474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0d02a748d54e5e97f80d163db5cbba, user_id=be877ff3bb304e589eaffa96709882f6, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.906555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da02f7-da89-4829-9112-fc5028d0d474 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.919730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1345/2688] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:52 2022] PUT /identity/v3/projects/7f0d02a748d54e5e97f80d163db5cbba/users/be877ff3bb304e589eaffa96709882f6/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:52.924377 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03985-ef7a-4761-8557-45d12e73833c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.934395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03985-ef7a-4761-8557-45d12e73833c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.994320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cce836-e65a-4f17-9b03-14b880b9acb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.994973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cce836-e65a-4f17-9b03-14b880b9acb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.995210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cce836-e65a-4f17-9b03-14b880b9acb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.995524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cce836-e65a-4f17-9b03-14b880b9acb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.996122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cce836-e65a-4f17-9b03-14b880b9acb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.997729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cce836-e65a-4f17-9b03-14b880b9acb2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:53.008868 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1344/2689] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:53.013385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da02f7-da89-4829-9112-fc5028d0d474 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.026940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da02f7-da89-4829-9112-fc5028d0d474 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.033080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed248336-34d9-4547-ab8e-dca489d3cf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.033785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed248336-34d9-4547-ab8e-dca489d3cf35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.034063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed248336-34d9-4547-ab8e-dca489d3cf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.034335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed248336-34d9-4547-ab8e-dca489d3cf35 admin admin] PATH_INFO: `/v3/projects/7f0d02a748d54e5e97f80d163db5cbba/users/be877ff3bb304e589eaffa96709882f6/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.036426 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed248336-34d9-4547-ab8e-dca489d3cf35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0d02a748d54e5e97f80d163db5cbba, user_id=be877ff3bb304e589eaffa96709882f6, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:53.039069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed248336-34d9-4547-ab8e-dca489d3cf35 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:53.051803 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1346/2690] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:53 2022] PUT /identity/v3/projects/7f0d02a748d54e5e97f80d163db5cbba/users/be877ff3bb304e589eaffa96709882f6/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:53.063024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b04b39-f33f-4001-a82c-fd8626a158dd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.063311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b04b39-f33f-4001-a82c-fd8626a158dd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.063589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b04b39-f33f-4001-a82c-fd8626a158dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.101932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-24b04b39-f33f-4001-a82c-fd8626a158dd None None] MFA Rules not processed for user `be877ff3bb304e589eaffa96709882f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:53.103156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-24b04b39-f33f-4001-a82c-fd8626a158dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:53.174748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1345/2691] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:53 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:53.186301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed248336-34d9-4547-ab8e-dca489d3cf35 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.188205 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ed248336-34d9-4547-ab8e-dca489d3cf35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:53.211490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:53.250165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed248336-34d9-4547-ab8e-dca489d3cf35 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.253784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e52ca-c431-40c9-8601-79138471f0ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.255026 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e52ca-c431-40c9-8601-79138471f0ab service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.255295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e52ca-c431-40c9-8601-79138471f0ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.255567 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e52ca-c431-40c9-8601-79138471f0ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.264588 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e52ca-c431-40c9-8601-79138471f0ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:53.266046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e52ca-c431-40c9-8601-79138471f0ab service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:53.275162 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1347/2692] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:53.322531 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b04b39-f33f-4001-a82c-fd8626a158dd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.323923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-24b04b39-f33f-4001-a82c-fd8626a158dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:53.336212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b04b39-f33f-4001-a82c-fd8626a158dd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.340041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57bf0b-a3b5-45ce-99b0-d641ea0675b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.340635 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57bf0b-a3b5-45ce-99b0-d641ea0675b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.340911 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57bf0b-a3b5-45ce-99b0-d641ea0675b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.341207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57bf0b-a3b5-45ce-99b0-d641ea0675b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.388552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:53.398662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57bf0b-a3b5-45ce-99b0-d641ea0675b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:53.400151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57bf0b-a3b5-45ce-99b0-d641ea0675b7 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:53.408923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1346/2693] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:56.804542 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e52ca-c431-40c9-8601-79138471f0ab service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.895107 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e52ca-c431-40c9-8601-79138471f0ab service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.905057 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.905756 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.906038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.906313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] PATH_INFO: `/v3/users/2d7a005fea2f4a83826da1fcbb64f4a6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.907467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d7a005fea2f4a83826da1fcbb64f4a6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.909506 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.959490 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d7a005fea2f4a83826da1fcbb64f4a6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:56.971979 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d7a005fea2f4a83826da1fcbb64f4a6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:56.979884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d7a005fea2f4a83826da1fcbb64f4a6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:56.989102 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d7a005fea2f4a83826da1fcbb64f4a6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:56.989913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d7a005fea2f4a83826da1fcbb64f4a6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:56.996058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1348/2694] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:56 2022] DELETE /identity/v3/users/2d7a005fea2f4a83826da1fcbb64f4a6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:57.000675 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf271e0a-c921-4436-8095-d549ecb2f19a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:57.001234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf271e0a-c921-4436-8095-d549ecb2f19a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:57.001234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf271e0a-c921-4436-8095-d549ecb2f19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.022170 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bf271e0a-c921-4436-8095-d549ecb2f19a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:57.023389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bf271e0a-c921-4436-8095-d549ecb2f19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:57.085637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1347/2695] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:57.095651 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:57.097147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:57.148613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:57.167115 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87d290-5f1b-494d-97e9-0418424d6505 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:57.172435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60a9f0-36d3-4945-95e6-8304f1fba591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:57.173051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60a9f0-36d3-4945-95e6-8304f1fba591 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:57.173326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60a9f0-36d3-4945-95e6-8304f1fba591 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:57.173617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60a9f0-36d3-4945-95e6-8304f1fba591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.183853 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60a9f0-36d3-4945-95e6-8304f1fba591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:57.185504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60a9f0-36d3-4945-95e6-8304f1fba591 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:57.196356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1349/2696] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:57.469463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf271e0a-c921-4436-8095-d549ecb2f19a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:57.471054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bf271e0a-c921-4436-8095-d549ecb2f19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:57.485098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf271e0a-c921-4436-8095-d549ecb2f19a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:57.490940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:57.491783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:57.492188 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:57.492460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] PATH_INFO: `/v3/projects/d9afbc1494cb467299917d0c17592840` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.494490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9afbc1494cb467299917d0c17592840)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:57.496666 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:57.548118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9afbc1494cb467299917d0c17592840', 'request_id': 'req-0bc0d819-3aab-4198-bc6f-79210df9d7a9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.554867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9afbc1494cb467299917d0c17592840', 'request_id': 'req-0bc0d819-3aab-4198-bc6f-79210df9d7a9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.560489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9afbc1494cb467299917d0c17592840', 'request_id': 'req-0bc0d819-3aab-4198-bc6f-79210df9d7a9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.573703 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] The token cache is being invalidate because project d9afbc1494cb467299917d0c17592840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:57.573988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.574665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.576234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1348/2697] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:57 2022] DELETE /identity/v3/projects/d9afbc1494cb467299917d0c17592840 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:57.580212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60a9f0-36d3-4945-95e6-8304f1fba591 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:57.590863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60a9f0-36d3-4945-95e6-8304f1fba591 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:57.647669 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:57.648297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:57.648573 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:57.648841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] PATH_INFO: `/v3/users/be877ff3bb304e589eaffa96709882f6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.649988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be877ff3bb304e589eaffa96709882f6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:57.651898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:57.698879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be877ff3bb304e589eaffa96709882f6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.710776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be877ff3bb304e589eaffa96709882f6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.718044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be877ff3bb304e589eaffa96709882f6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.729088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be877ff3bb304e589eaffa96709882f6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.729872 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a8732f6f-c55f-432e-986c-d4e13aaef795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be877ff3bb304e589eaffa96709882f6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:57.736476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1350/2698] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:44:57 2022] DELETE /identity/v3/users/be877ff3bb304e589eaffa96709882f6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:58.057022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.058538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:58.102895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0d819-3aab-4198-bc6f-79210df9d7a9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.169019 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.169687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.169969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.170243 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] PATH_INFO: `/v3/projects/7f0d02a748d54e5e97f80d163db5cbba` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.171359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0d02a748d54e5e97f80d163db5cbba)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.173237 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.194953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:58.231235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0d02a748d54e5e97f80d163db5cbba', 'request_id': 'req-e61a194b-c4e6-46b7-9af0-d33ed2295b53'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:58.239535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0d02a748d54e5e97f80d163db5cbba', 'request_id': 'req-e61a194b-c4e6-46b7-9af0-d33ed2295b53'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:58.245174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0d02a748d54e5e97f80d163db5cbba', 'request_id': 'req-e61a194b-c4e6-46b7-9af0-d33ed2295b53'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:58.260692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] The token cache is being invalidate because project 7f0d02a748d54e5e97f80d163db5cbba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:44:58.261027 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:58.261723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e61a194b-c4e6-46b7-9af0-d33ed2295b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:44:58.265610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1349/2699] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:44:58 2022] DELETE /identity/v3/projects/7f0d02a748d54e5e97f80d163db5cbba => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:58.268032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41a989-d81a-4192-a147-b5f49d1bc2fa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.268504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41a989-d81a-4192-a147-b5f49d1bc2fa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.268594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41a989-d81a-4192-a147-b5f49d1bc2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.293256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5b41a989-d81a-4192-a147-b5f49d1bc2fa None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:58.294564 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5b41a989-d81a-4192-a147-b5f49d1bc2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:58.358607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1351/2700] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:58.367080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e863a5-b9d3-4f11-add3-493a143aedeb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.367395 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e863a5-b9d3-4f11-add3-493a143aedeb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.367629 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e863a5-b9d3-4f11-add3-493a143aedeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.388833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b2e863a5-b9d3-4f11-add3-493a143aedeb None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:58.389408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b2e863a5-b9d3-4f11-add3-493a143aedeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:58.399247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1350/2701] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:44:58.403705 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41a989-d81a-4192-a147-b5f49d1bc2fa None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.408549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:58.414980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41a989-d81a-4192-a147-b5f49d1bc2fa None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.420975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9f153-ebba-4386-8110-5952df73a3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.421591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f153-ebba-4386-8110-5952df73a3ab admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.421873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f153-ebba-4386-8110-5952df73a3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.422143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f153-ebba-4386-8110-5952df73a3ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.422787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9f153-ebba-4386-8110-5952df73a3ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:58.423079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9f153-ebba-4386-8110-5952df73a3ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.424637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9f153-ebba-4386-8110-5952df73a3ab admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.437938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1352/2702] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:44:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:58.442942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e863a5-b9d3-4f11-add3-493a143aedeb None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.452582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e863a5-b9d3-4f11-add3-493a143aedeb None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.457438 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f6ae9-7996-465f-9b25-38f53b1d61e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.458079 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f6ae9-7996-465f-9b25-38f53b1d61e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.458356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f6ae9-7996-465f-9b25-38f53b1d61e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.458628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f6ae9-7996-465f-9b25-38f53b1d61e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.459284 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f6ae9-7996-465f-9b25-38f53b1d61e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.461174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f6ae9-7996-465f-9b25-38f53b1d61e7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.489516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1351/2703] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:58 2022] POST /identity/v3/projects => generated 384 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:58.495140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9f153-ebba-4386-8110-5952df73a3ab admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.507189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9f153-ebba-4386-8110-5952df73a3ab admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.572031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a2810-f65f-43c4-82ac-076fa222209a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.572636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a2810-f65f-43c4-82ac-076fa222209a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.572940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a2810-f65f-43c4-82ac-076fa222209a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.573189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a2810-f65f-43c4-82ac-076fa222209a admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.573850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a2810-f65f-43c4-82ac-076fa222209a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.575765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a2810-f65f-43c4-82ac-076fa222209a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.598524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1353/2704] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:58 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:58.602476 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f6ae9-7996-465f-9b25-38f53b1d61e7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.612759 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f6ae9-7996-465f-9b25-38f53b1d61e7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.618436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103768c-8580-4e3c-8085-6f03baef9bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.619014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103768c-8580-4e3c-8085-6f03baef9bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.619294 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103768c-8580-4e3c-8085-6f03baef9bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.619565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103768c-8580-4e3c-8085-6f03baef9bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.620205 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103768c-8580-4e3c-8085-6f03baef9bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.621856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103768c-8580-4e3c-8085-6f03baef9bc4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.632729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1352/2705] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:58 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:58.636939 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a2810-f65f-43c4-82ac-076fa222209a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.647201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a2810-f65f-43c4-82ac-076fa222209a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.652472 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33a2ed-44bd-4ac9-a843-ef10d596f1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.653170 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a2ed-44bd-4ac9-a843-ef10d596f1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.653883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a2ed-44bd-4ac9-a843-ef10d596f1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.653883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a2ed-44bd-4ac9-a843-ef10d596f1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.654549 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33a2ed-44bd-4ac9-a843-ef10d596f1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.656249 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33a2ed-44bd-4ac9-a843-ef10d596f1a8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.669437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1354/2706] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:58 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:58.673989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103768c-8580-4e3c-8085-6f03baef9bc4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.684972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103768c-8580-4e3c-8085-6f03baef9bc4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.689797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c8ac5-9101-4aaf-936b-4094b43784bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.690467 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c8ac5-9101-4aaf-936b-4094b43784bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.690747 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c8ac5-9101-4aaf-936b-4094b43784bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.691022 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c8ac5-9101-4aaf-936b-4094b43784bd admin admin] PATH_INFO: `/v3/projects/4d11009b1b88480293a9599eefd77388/users/5451ca55fb61427e9ab71dd73853a7bb/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.706731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c8ac5-9101-4aaf-936b-4094b43784bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d11009b1b88480293a9599eefd77388, user_id=5451ca55fb61427e9ab71dd73853a7bb, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.709478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c8ac5-9101-4aaf-936b-4094b43784bd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.723382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1353/2707] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:58 2022] PUT /identity/v3/projects/4d11009b1b88480293a9599eefd77388/users/5451ca55fb61427e9ab71dd73853a7bb/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:58.735828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81b666-2d03-496a-ac9c-4e641ce96975 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.736111 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81b666-2d03-496a-ac9c-4e641ce96975 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.736386 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81b666-2d03-496a-ac9c-4e641ce96975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.764628 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2c81b666-2d03-496a-ac9c-4e641ce96975 None None] MFA Rules not processed for user `5451ca55fb61427e9ab71dd73853a7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:58.765864 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2c81b666-2d03-496a-ac9c-4e641ce96975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:58.829943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1355/2708] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:58.834760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c8ac5-9101-4aaf-936b-4094b43784bd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.895583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c8ac5-9101-4aaf-936b-4094b43784bd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.900253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5958-0d78-478a-bf95-5b02f5f7f0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.900848 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5958-0d78-478a-bf95-5b02f5f7f0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.901131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5958-0d78-478a-bf95-5b02f5f7f0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.901403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5958-0d78-478a-bf95-5b02f5f7f0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.902057 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f5958-0d78-478a-bf95-5b02f5f7f0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.903934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f5958-0d78-478a-bf95-5b02f5f7f0f9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.931245 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1354/2709] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:44:58 2022] POST /identity/v3/projects => generated 384 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:58.935558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81b666-2d03-496a-ac9c-4e641ce96975 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.946254 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81b666-2d03-496a-ac9c-4e641ce96975 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.999271 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:59.006804 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec408b6-b702-46e0-a81b-858f91d2cb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.007585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec408b6-b702-46e0-a81b-858f91d2cb7e admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.008031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec408b6-b702-46e0-a81b-858f91d2cb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.008308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec408b6-b702-46e0-a81b-858f91d2cb7e admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.008944 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec408b6-b702-46e0-a81b-858f91d2cb7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.010885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec408b6-b702-46e0-a81b-858f91d2cb7e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.032075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1356/2710] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:44:58 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:59.036670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5958-0d78-478a-bf95-5b02f5f7f0f9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.047145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5958-0d78-478a-bf95-5b02f5f7f0f9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.052346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143d2c8-1aa2-4d4a-a6b1-9638cd486687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.052934 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143d2c8-1aa2-4d4a-a6b1-9638cd486687 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.053211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143d2c8-1aa2-4d4a-a6b1-9638cd486687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.053481 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143d2c8-1aa2-4d4a-a6b1-9638cd486687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.054143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143d2c8-1aa2-4d4a-a6b1-9638cd486687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.055714 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143d2c8-1aa2-4d4a-a6b1-9638cd486687 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.066044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1355/2711] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:59 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:59.070216 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec408b6-b702-46e0-a81b-858f91d2cb7e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.080143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec408b6-b702-46e0-a81b-858f91d2cb7e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.084748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb704e4-5f69-43a5-9e84-a541ef20a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.085387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb704e4-5f69-43a5-9e84-a541ef20a4cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.085698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb704e4-5f69-43a5-9e84-a541ef20a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.085972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb704e4-5f69-43a5-9e84-a541ef20a4cc admin admin] PATH_INFO: `/v3/projects/328ba0528de244fba9658312ae024cbd/users/b51bf3ba043a4db5b1d5cde54ff2db09/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.100926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb704e4-5f69-43a5-9e84-a541ef20a4cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=328ba0528de244fba9658312ae024cbd, user_id=b51bf3ba043a4db5b1d5cde54ff2db09, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.103731 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb704e4-5f69-43a5-9e84-a541ef20a4cc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.123342 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1357/2712] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:59 2022] PUT /identity/v3/projects/328ba0528de244fba9658312ae024cbd/users/b51bf3ba043a4db5b1d5cde54ff2db09/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:59.127332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143d2c8-1aa2-4d4a-a6b1-9638cd486687 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.137443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143d2c8-1aa2-4d4a-a6b1-9638cd486687 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.149353 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:59.200497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6d0aa-2b3a-44c9-9208-efd008636a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.201204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6d0aa-2b3a-44c9-9208-efd008636a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.201492 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6d0aa-2b3a-44c9-9208-efd008636a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.201790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6d0aa-2b3a-44c9-9208-efd008636a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.202437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6d0aa-2b3a-44c9-9208-efd008636a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.204048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6d0aa-2b3a-44c9-9208-efd008636a67 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.215246 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1356/2713] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:44:59 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:44:59.219464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb704e4-5f69-43a5-9e84-a541ef20a4cc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.230687 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb704e4-5f69-43a5-9e84-a541ef20a4cc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.235521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12b1e6-e68e-42f9-b668-828a0ffdebdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.236259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12b1e6-e68e-42f9-b668-828a0ffdebdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.236605 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12b1e6-e68e-42f9-b668-828a0ffdebdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.236943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12b1e6-e68e-42f9-b668-828a0ffdebdf admin admin] PATH_INFO: `/v3/projects/328ba0528de244fba9658312ae024cbd/users/b51bf3ba043a4db5b1d5cde54ff2db09/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.240225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12b1e6-e68e-42f9-b668-828a0ffdebdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=328ba0528de244fba9658312ae024cbd, user_id=b51bf3ba043a4db5b1d5cde54ff2db09, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.242973 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12b1e6-e68e-42f9-b668-828a0ffdebdf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.254793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1358/2714] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:44:59 2022] PUT /identity/v3/projects/328ba0528de244fba9658312ae024cbd/users/b51bf3ba043a4db5b1d5cde54ff2db09/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:59.265355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bc1e2-cd59-43a5-a9f2-8d95b96bc84a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.265864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bc1e2-cd59-43a5-a9f2-8d95b96bc84a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.266144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bc1e2-cd59-43a5-a9f2-8d95b96bc84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.296991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ad9bc1e2-cd59-43a5-a9f2-8d95b96bc84a None None] MFA Rules not processed for user `b51bf3ba043a4db5b1d5cde54ff2db09`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:59.298257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ad9bc1e2-cd59-43a5-a9f2-8d95b96bc84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:59.366893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1357/2715] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:44:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:44:59.377540 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12b1e6-e68e-42f9-b668-828a0ffdebdf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.379976 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6c12b1e6-e68e-42f9-b668-828a0ffdebdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:59.458219 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12b1e6-e68e-42f9-b668-828a0ffdebdf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.461816 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bdd30-ab84-4e84-a6aa-a3ecc24e69a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.462416 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bdd30-ab84-4e84-a6aa-a3ecc24e69a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.462695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bdd30-ab84-4e84-a6aa-a3ecc24e69a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.462967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bdd30-ab84-4e84-a6aa-a3ecc24e69a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.471978 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bdd30-ab84-4e84-a6aa-a3ecc24e69a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.474038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bdd30-ab84-4e84-a6aa-a3ecc24e69a6 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.483370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1359/2716] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:44:59.588504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bc1e2-cd59-43a5-a9f2-8d95b96bc84a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.599131 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bc1e2-cd59-43a5-a9f2-8d95b96bc84a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.602865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aee133-77fc-4226-ab21-73a13ba9600c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.603489 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aee133-77fc-4226-ab21-73a13ba9600c service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.603763 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aee133-77fc-4226-ab21-73a13ba9600c service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.604156 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aee133-77fc-4226-ab21-73a13ba9600c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.672568 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aee133-77fc-4226-ab21-73a13ba9600c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.674049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aee133-77fc-4226-ab21-73a13ba9600c service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.684051 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1358/2717] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:44:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:00.745098 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bdd30-ab84-4e84-a6aa-a3ecc24e69a6 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.746599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-887bdd30-ab84-4e84-a6aa-a3ecc24e69a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:00.814407 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bdd30-ab84-4e84-a6aa-a3ecc24e69a6 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:00.820028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:00.820654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.820932 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.821198 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] PATH_INFO: `/v3/users/8afd7b912a4f45af80765383201f1c70` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.822427 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8afd7b912a4f45af80765383201f1c70)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:00.824308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:00.861585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:00.873121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8afd7b912a4f45af80765383201f1c70'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:00.885772 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8afd7b912a4f45af80765383201f1c70'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:00.892467 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8afd7b912a4f45af80765383201f1c70'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:00.901360 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8afd7b912a4f45af80765383201f1c70'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:00.902024 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8afd7b912a4f45af80765383201f1c70'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:00.908031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1360/2718] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:00 2022] DELETE /identity/v3/users/8afd7b912a4f45af80765383201f1c70 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:01.087164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aee133-77fc-4226-ab21-73a13ba9600c service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.099790 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aee133-77fc-4226-ab21-73a13ba9600c service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.136003 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.163755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.164457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.164731 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.164994 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] PATH_INFO: `/v3/projects/fb94475797a3463093c30f61eff2193a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.166631 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb94475797a3463093c30f61eff2193a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.168543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.227750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb94475797a3463093c30f61eff2193a', 'request_id': 'req-9d7be4b8-7aeb-4f8e-ace8-696c85229555'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.234304 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb94475797a3463093c30f61eff2193a', 'request_id': 'req-9d7be4b8-7aeb-4f8e-ace8-696c85229555'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.241031 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb94475797a3463093c30f61eff2193a', 'request_id': 'req-9d7be4b8-7aeb-4f8e-ace8-696c85229555'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.255557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] The token cache is being invalidate because project fb94475797a3463093c30f61eff2193a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:01.255892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.256607 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.258598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1359/2719] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:01 2022] DELETE /identity/v3/projects/fb94475797a3463093c30f61eff2193a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:01.263064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.264523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:01.278474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdc8c0-decf-4f46-acd1-1e5dbaf98087 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.336041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.336623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.336902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.337173 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] PATH_INFO: `/v3/users/ce46630863374be6895ec73eebfba9cf` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.338401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce46630863374be6895ec73eebfba9cf)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.340380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.390683 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce46630863374be6895ec73eebfba9cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.403279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce46630863374be6895ec73eebfba9cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.411070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce46630863374be6895ec73eebfba9cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.418580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce46630863374be6895ec73eebfba9cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.419388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b87338cb-ac94-4b17-a694-cc45574615da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce46630863374be6895ec73eebfba9cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.426688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1361/2720] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:01 2022] DELETE /identity/v3/users/ce46630863374be6895ec73eebfba9cf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:01.604515 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.606089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:01.619651 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7be4b8-7aeb-4f8e-ace8-696c85229555 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.680609 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.681261 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.681540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.681838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] PATH_INFO: `/v3/projects/c1b2bf6fa8bd4787a32659e335949dc3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.682940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b2bf6fa8bd4787a32659e335949dc3)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.684854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.729852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b2bf6fa8bd4787a32659e335949dc3', 'request_id': 'req-dc9984d1-352b-4dbc-853f-508a81134147'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.735916 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b2bf6fa8bd4787a32659e335949dc3', 'request_id': 'req-dc9984d1-352b-4dbc-853f-508a81134147'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.742068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b2bf6fa8bd4787a32659e335949dc3', 'request_id': 'req-dc9984d1-352b-4dbc-853f-508a81134147'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.745730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.758308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] The token cache is being invalidate because project c1b2bf6fa8bd4787a32659e335949dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:01.758599 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.759240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dc9984d1-352b-4dbc-853f-508a81134147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:01.760753 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1360/2721] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:01 2022] DELETE /identity/v3/projects/c1b2bf6fa8bd4787a32659e335949dc3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:01.765878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c03657-af4c-4b85-8018-dbfaaf0bb9d1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.766355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c03657-af4c-4b85-8018-dbfaaf0bb9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.766557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c03657-af4c-4b85-8018-dbfaaf0bb9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.786000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e4c03657-af4c-4b85-8018-dbfaaf0bb9d1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:01.787197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e4c03657-af4c-4b85-8018-dbfaaf0bb9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:01.813870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.850005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1362/2722] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:01.858827 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ca4ff-226f-4bb1-a86e-cf7c5ff9a3e8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.859111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ca4ff-226f-4bb1-a86e-cf7c5ff9a3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.859385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ca4ff-226f-4bb1-a86e-cf7c5ff9a3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.880278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5f8ca4ff-226f-4bb1-a86e-cf7c5ff9a3e8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:01.881497 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5f8ca4ff-226f-4bb1-a86e-cf7c5ff9a3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:01.889661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1361/2723] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:01.894108 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c03657-af4c-4b85-8018-dbfaaf0bb9d1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.904051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c03657-af4c-4b85-8018-dbfaaf0bb9d1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.908676 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133b124-9b4e-48e4-b370-18959cc53f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.909270 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133b124-9b4e-48e4-b370-18959cc53f0c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.909553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133b124-9b4e-48e4-b370-18959cc53f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.909855 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133b124-9b4e-48e4-b370-18959cc53f0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.911000 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133b124-9b4e-48e4-b370-18959cc53f0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:01.911313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133b124-9b4e-48e4-b370-18959cc53f0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.912890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133b124-9b4e-48e4-b370-18959cc53f0c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.924644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1363/2724] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:45:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:01.929902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ca4ff-226f-4bb1-a86e-cf7c5ff9a3e8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.939095 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ca4ff-226f-4bb1-a86e-cf7c5ff9a3e8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.943946 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00996dfa-77a6-448c-ab1c-4d1a715bb6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.944532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00996dfa-77a6-448c-ab1c-4d1a715bb6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.944805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00996dfa-77a6-448c-ab1c-4d1a715bb6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.945067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00996dfa-77a6-448c-ab1c-4d1a715bb6c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.945768 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00996dfa-77a6-448c-ab1c-4d1a715bb6c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.947614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00996dfa-77a6-448c-ab1c-4d1a715bb6c9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.975805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1362/2725] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:01 2022] POST /identity/v3/projects => generated 384 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:01.980122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133b124-9b4e-48e4-b370-18959cc53f0c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.991140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133b124-9b4e-48e4-b370-18959cc53f0c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.046695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64982e4-6b14-4d41-ae5e-ada1c776b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.047299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64982e4-6b14-4d41-ae5e-ada1c776b2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.047581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64982e4-6b14-4d41-ae5e-ada1c776b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.047856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64982e4-6b14-4d41-ae5e-ada1c776b2e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.048485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64982e4-6b14-4d41-ae5e-ada1c776b2e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.050432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64982e4-6b14-4d41-ae5e-ada1c776b2e6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.072201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1364/2726] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:01 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:02.076330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00996dfa-77a6-448c-ab1c-4d1a715bb6c9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.086785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00996dfa-77a6-448c-ab1c-4d1a715bb6c9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.091602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ada900-e45a-4135-a4cf-c75d7931a925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.092185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ada900-e45a-4135-a4cf-c75d7931a925 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.092454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ada900-e45a-4135-a4cf-c75d7931a925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.092720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ada900-e45a-4135-a4cf-c75d7931a925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.093346 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ada900-e45a-4135-a4cf-c75d7931a925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.095209 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ada900-e45a-4135-a4cf-c75d7931a925 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.106598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1363/2727] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:45:02.110523 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64982e4-6b14-4d41-ae5e-ada1c776b2e6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.123469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64982e4-6b14-4d41-ae5e-ada1c776b2e6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.128103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4e52f-8a53-4c56-bb83-e0cb8127fd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.128680 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4e52f-8a53-4c56-bb83-e0cb8127fd79 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.129654 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4e52f-8a53-4c56-bb83-e0cb8127fd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.129934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4e52f-8a53-4c56-bb83-e0cb8127fd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.130561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4e52f-8a53-4c56-bb83-e0cb8127fd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.132140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4e52f-8a53-4c56-bb83-e0cb8127fd79 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.143725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1365/2728] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:45:02.149108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ada900-e45a-4135-a4cf-c75d7931a925 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.159392 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ada900-e45a-4135-a4cf-c75d7931a925 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.164935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8bb8-7154-47c6-a6e8-b5e51c78c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.165582 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8bb8-7154-47c6-a6e8-b5e51c78c4ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.166122 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8bb8-7154-47c6-a6e8-b5e51c78c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.166391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8bb8-7154-47c6-a6e8-b5e51c78c4ee admin admin] PATH_INFO: `/v3/projects/0a40a06f36814d91a69f580ab398d3c7/users/830b7b41b0b149de8c3fb3b33ef56b94/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.181424 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8bb8-7154-47c6-a6e8-b5e51c78c4ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a40a06f36814d91a69f580ab398d3c7, user_id=830b7b41b0b149de8c3fb3b33ef56b94, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.184069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8bb8-7154-47c6-a6e8-b5e51c78c4ee admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.199213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1364/2729] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:02 2022] PUT /identity/v3/projects/0a40a06f36814d91a69f580ab398d3c7/users/830b7b41b0b149de8c3fb3b33ef56b94/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:02.210550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdf478-445c-49cc-bc4d-a621f8a5d268 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.210829 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdf478-445c-49cc-bc4d-a621f8a5d268 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.211099 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdf478-445c-49cc-bc4d-a621f8a5d268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.239413 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8bcdf478-445c-49cc-bc4d-a621f8a5d268 None None] MFA Rules not processed for user `830b7b41b0b149de8c3fb3b33ef56b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:02.240606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8bcdf478-445c-49cc-bc4d-a621f8a5d268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:02.307570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1366/2730] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:45:02.312028 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8bb8-7154-47c6-a6e8-b5e51c78c4ee admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.321891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdf478-445c-49cc-bc4d-a621f8a5d268 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.324076 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8bcdf478-445c-49cc-bc4d-a621f8a5d268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:02.356927 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:02.376719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8bb8-7154-47c6-a6e8-b5e51c78c4ee admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.381664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d73b37-8a2b-4a2b-95b4-d24d1aafddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.382260 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d73b37-8a2b-4a2b-95b4-d24d1aafddd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.382540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d73b37-8a2b-4a2b-95b4-d24d1aafddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.382817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d73b37-8a2b-4a2b-95b4-d24d1aafddd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.383454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d73b37-8a2b-4a2b-95b4-d24d1aafddd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.385330 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d73b37-8a2b-4a2b-95b4-d24d1aafddd7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.393476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdf478-445c-49cc-bc4d-a621f8a5d268 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.398884 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.399511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.399786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.400054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] PATH_INFO: `/v3/users/5451ca55fb61427e9ab71dd73853a7bb` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.401172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5451ca55fb61427e9ab71dd73853a7bb)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.403273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.415163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1365/2731] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:02 2022] POST /identity/v3/projects => generated 386 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:02.419543 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d73b37-8a2b-4a2b-95b4-d24d1aafddd7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.456381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5451ca55fb61427e9ab71dd73853a7bb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:02.468720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5451ca55fb61427e9ab71dd73853a7bb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:02.475883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5451ca55fb61427e9ab71dd73853a7bb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:02.483727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5451ca55fb61427e9ab71dd73853a7bb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:02.484608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d73b37-8a2b-4a2b-95b4-d24d1aafddd7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.484741 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73f860c9-ef39-423e-9562-5189fe70ce3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5451ca55fb61427e9ab71dd73853a7bb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:02.490546 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142b604-4793-4675-bc4e-6042bfb8c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.491151 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142b604-4793-4675-bc4e-6042bfb8c92f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.491431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142b604-4793-4675-bc4e-6042bfb8c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.491701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142b604-4793-4675-bc4e-6042bfb8c92f admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.491801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1367/2732] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:02 2022] DELETE /identity/v3/users/5451ca55fb61427e9ab71dd73853a7bb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:02.492327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142b604-4793-4675-bc4e-6042bfb8c92f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.494285 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142b604-4793-4675-bc4e-6042bfb8c92f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.496365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ad5da-4e34-4df3-af6f-777ff120589f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.496652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ad5da-4e34-4df3-af6f-777ff120589f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.496926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ad5da-4e34-4df3-af6f-777ff120589f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.518091 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1366/2733] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:02 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:45:02.522383 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142b604-4793-4675-bc4e-6042bfb8c92f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.522924 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1e2ad5da-4e34-4df3-af6f-777ff120589f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:02.524107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f142b604-4793-4675-bc4e-6042bfb8c92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:02.524237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1e2ad5da-4e34-4df3-af6f-777ff120589f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:02.537107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142b604-4793-4675-bc4e-6042bfb8c92f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.541414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:02.590167 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1368/2734] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:45:02.598462 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae22af-0130-419f-bea0-a8d46661fc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.599111 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae22af-0130-419f-bea0-a8d46661fc99 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.599396 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae22af-0130-419f-bea0-a8d46661fc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.599748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae22af-0130-419f-bea0-a8d46661fc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.600400 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae22af-0130-419f-bea0-a8d46661fc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.600846 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ad5da-4e34-4df3-af6f-777ff120589f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.602020 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae22af-0130-419f-bea0-a8d46661fc99 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.602262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1e2ad5da-4e34-4df3-af6f-777ff120589f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:02.613157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1367/2735] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:45:02.617695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae22af-0130-419f-bea0-a8d46661fc99 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.627648 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae22af-0130-419f-bea0-a8d46661fc99 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.633318 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efe354-f22c-439d-a63d-099b5883ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.633953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efe354-f22c-439d-a63d-099b5883ecf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.634234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efe354-f22c-439d-a63d-099b5883ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.634506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efe354-f22c-439d-a63d-099b5883ecf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.635166 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efe354-f22c-439d-a63d-099b5883ecf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.636749 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efe354-f22c-439d-a63d-099b5883ecf0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.648728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1368/2736] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:45:02.653068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efe354-f22c-439d-a63d-099b5883ecf0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.661322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:02.666595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efe354-f22c-439d-a63d-099b5883ecf0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.668310 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ad5da-4e34-4df3-af6f-777ff120589f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.671704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a08d5a-fc5e-4fcc-bd34-369a38b7b97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.672365 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a08d5a-fc5e-4fcc-bd34-369a38b7b97e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.672637 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a08d5a-fc5e-4fcc-bd34-369a38b7b97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.672906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a08d5a-fc5e-4fcc-bd34-369a38b7b97e admin admin] PATH_INFO: `/v3/projects/99c96206d9994dc89ed2f79fc8ac4cab/users/4318aeb27c124372aeb3e35f982af736/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.673771 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f1db9-3499-4524-995b-9a2697fd4592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.674380 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f1db9-3499-4524-995b-9a2697fd4592 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.674685 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f1db9-3499-4524-995b-9a2697fd4592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.674956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f1db9-3499-4524-995b-9a2697fd4592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.686177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f1db9-3499-4524-995b-9a2697fd4592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.687591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f1db9-3499-4524-995b-9a2697fd4592 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.689054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a08d5a-fc5e-4fcc-bd34-369a38b7b97e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c96206d9994dc89ed2f79fc8ac4cab, user_id=4318aeb27c124372aeb3e35f982af736, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.691962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a08d5a-fc5e-4fcc-bd34-369a38b7b97e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.698681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1369/2737] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:02.705672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1369/2738] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:02 2022] PUT /identity/v3/projects/99c96206d9994dc89ed2f79fc8ac4cab/users/4318aeb27c124372aeb3e35f982af736/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:02.717453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed51d3-219c-41dc-959d-06c4764c5546 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.717788 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed51d3-219c-41dc-959d-06c4764c5546 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.718064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed51d3-219c-41dc-959d-06c4764c5546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.748384 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-75ed51d3-219c-41dc-959d-06c4764c5546 None None] MFA Rules not processed for user `4318aeb27c124372aeb3e35f982af736`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:02.749727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-75ed51d3-219c-41dc-959d-06c4764c5546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:02.819279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1370/2739] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:02.823929 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a08d5a-fc5e-4fcc-bd34-369a38b7b97e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.898975 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a08d5a-fc5e-4fcc-bd34-369a38b7b97e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.903697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006daa9-58ef-49df-95ad-edea61b25d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.904303 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006daa9-58ef-49df-95ad-edea61b25d0e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.904580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006daa9-58ef-49df-95ad-edea61b25d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.904874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006daa9-58ef-49df-95ad-edea61b25d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.905501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006daa9-58ef-49df-95ad-edea61b25d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.907410 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006daa9-58ef-49df-95ad-edea61b25d0e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.935551 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1370/2740] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:02 2022] POST /identity/v3/projects => generated 384 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:45:02.939821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed51d3-219c-41dc-959d-06c4764c5546 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.951814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed51d3-219c-41dc-959d-06c4764c5546 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.996962 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006daa9-58ef-49df-95ad-edea61b25d0e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.998979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5006daa9-58ef-49df-95ad-edea61b25d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:03.031373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f97adb-6377-4e59-bcca-ed9d1c3c64df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.032014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f97adb-6377-4e59-bcca-ed9d1c3c64df admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.032926 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f97adb-6377-4e59-bcca-ed9d1c3c64df admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.033202 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f97adb-6377-4e59-bcca-ed9d1c3c64df admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.033923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f97adb-6377-4e59-bcca-ed9d1c3c64df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.036819 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f97adb-6377-4e59-bcca-ed9d1c3c64df admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.066067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1371/2741] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:02 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:03.067785 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f97adb-6377-4e59-bcca-ed9d1c3c64df admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.071780 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:03.078468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f97adb-6377-4e59-bcca-ed9d1c3c64df admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.084038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e854e76-1359-4840-bbee-0191728e04db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.085023 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e854e76-1359-4840-bbee-0191728e04db admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.085329 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e854e76-1359-4840-bbee-0191728e04db admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.085638 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e854e76-1359-4840-bbee-0191728e04db admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.086294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e854e76-1359-4840-bbee-0191728e04db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.087895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e854e76-1359-4840-bbee-0191728e04db admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.090429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006daa9-58ef-49df-95ad-edea61b25d0e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.096884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.098086 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.098368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.098663 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] PATH_INFO: `/v3/projects/4d11009b1b88480293a9599eefd77388` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.098867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1372/2742] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:03 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:45:03.102581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d11009b1b88480293a9599eefd77388)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.102581 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.103776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e854e76-1359-4840-bbee-0191728e04db admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.119089 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e854e76-1359-4840-bbee-0191728e04db admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.124947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8a462-8b0d-4b72-8f32-6531e987c515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.126097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8a462-8b0d-4b72-8f32-6531e987c515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.126598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8a462-8b0d-4b72-8f32-6531e987c515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.127059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8a462-8b0d-4b72-8f32-6531e987c515 admin admin] PATH_INFO: `/v3/projects/ca2fecebb657455b80d83fb2a6638d9b/users/8cdca3c796f8495cb1f5861f59ec1365/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.147351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8a462-8b0d-4b72-8f32-6531e987c515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2fecebb657455b80d83fb2a6638d9b, user_id=8cdca3c796f8495cb1f5861f59ec1365, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.150189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8a462-8b0d-4b72-8f32-6531e987c515 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.159905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d11009b1b88480293a9599eefd77388', 'request_id': 'req-22bad5a9-d191-4392-8429-0d0edd20f0bd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.164883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1373/2743] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:03 2022] PUT /identity/v3/projects/ca2fecebb657455b80d83fb2a6638d9b/users/8cdca3c796f8495cb1f5861f59ec1365/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:45:03.167368 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d11009b1b88480293a9599eefd77388', 'request_id': 'req-22bad5a9-d191-4392-8429-0d0edd20f0bd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.169043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8a462-8b0d-4b72-8f32-6531e987c515 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.173793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d11009b1b88480293a9599eefd77388', 'request_id': 'req-22bad5a9-d191-4392-8429-0d0edd20f0bd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.190711 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] The token cache is being invalidate because project 4d11009b1b88480293a9599eefd77388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:03.190999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.192338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.194060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1371/2744] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:02 2022] DELETE /identity/v3/projects/4d11009b1b88480293a9599eefd77388 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:45:03.198821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.200254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:03.216779 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:03.248835 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8a462-8b0d-4b72-8f32-6531e987c515 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.254655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb729cd-6750-493b-8534-3e74d98d3903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.255275 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb729cd-6750-493b-8534-3e74d98d3903 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.255559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb729cd-6750-493b-8534-3e74d98d3903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.255860 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb729cd-6750-493b-8534-3e74d98d3903 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.256556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb729cd-6750-493b-8534-3e74d98d3903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.258404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb729cd-6750-493b-8534-3e74d98d3903 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.270414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1374/2745] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:03 2022] GET /identity/v3/roles => generated 1427 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:45:03.275058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb729cd-6750-493b-8534-3e74d98d3903 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.275378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bad5a9-d191-4392-8429-0d0edd20f0bd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.276689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aeb729cd-6750-493b-8534-3e74d98d3903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:03.280538 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.281277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.281767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.282222 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] PATH_INFO: `/v3/users/b51bf3ba043a4db5b1d5cde54ff2db09` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.283408 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b51bf3ba043a4db5b1d5cde54ff2db09)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.285893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.291920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb729cd-6750-493b-8534-3e74d98d3903 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.296914 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb90d8-52f7-441d-8b2b-56a140d96312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.297695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb90d8-52f7-441d-8b2b-56a140d96312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.297913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb90d8-52f7-441d-8b2b-56a140d96312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.298144 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb90d8-52f7-441d-8b2b-56a140d96312 admin admin] PATH_INFO: `/v3/projects/ca2fecebb657455b80d83fb2a6638d9b/users/8cdca3c796f8495cb1f5861f59ec1365/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.300148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb90d8-52f7-441d-8b2b-56a140d96312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2fecebb657455b80d83fb2a6638d9b, user_id=8cdca3c796f8495cb1f5861f59ec1365, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.303444 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb90d8-52f7-441d-8b2b-56a140d96312 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.317662 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1375/2746] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:03 2022] PUT /identity/v3/projects/ca2fecebb657455b80d83fb2a6638d9b/users/8cdca3c796f8495cb1f5861f59ec1365/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:45:03.347193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b51bf3ba043a4db5b1d5cde54ff2db09'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.357767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b51bf3ba043a4db5b1d5cde54ff2db09'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.358540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b51bf3ba043a4db5b1d5cde54ff2db09'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.376017 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b51bf3ba043a4db5b1d5cde54ff2db09'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.384808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b51bf3ba043a4db5b1d5cde54ff2db09'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.389477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0c136-c004-4c72-9428-3efa6553c560 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.389712 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0c136-c004-4c72-9428-3efa6553c560 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.389991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0c136-c004-4c72-9428-3efa6553c560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.391364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1372/2747] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:03 2022] DELETE /identity/v3/users/b51bf3ba043a4db5b1d5cde54ff2db09 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:03.425059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f5a0c136-c004-4c72-9428-3efa6553c560 None None] MFA Rules not processed for user `8cdca3c796f8495cb1f5861f59ec1365`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:03.427906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f5a0c136-c004-4c72-9428-3efa6553c560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:03.519679 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1376/2748] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:03 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:45:03.532614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.534106 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:03.541479 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:03.634519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ce79-aa0b-4b05-bbfd-2478d5936c6d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.638791 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf8f1-920f-4f4a-9fe7-35e08fdb1e1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.639873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf8f1-920f-4f4a-9fe7-35e08fdb1e1e service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.640268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf8f1-920f-4f4a-9fe7-35e08fdb1e1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.640618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf8f1-920f-4f4a-9fe7-35e08fdb1e1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.642940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a25bf8f1-920f-4f4a-9fe7-35e08fdb1e1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:03.708061 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0c136-c004-4c72-9428-3efa6553c560 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.710295 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f5a0c136-c004-4c72-9428-3efa6553c560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:03.738074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bf8f1-920f-4f4a-9fe7-35e08fdb1e1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.739869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bf8f1-920f-4f4a-9fe7-35e08fdb1e1e service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.752453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1373/2749] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:03.816094 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0c136-c004-4c72-9428-3efa6553c560 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.821526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.822262 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.822493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.822769 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] PATH_INFO: `/v3/projects/328ba0528de244fba9658312ae024cbd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.824113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=328ba0528de244fba9658312ae024cbd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.826114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.876907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:03.881212 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328ba0528de244fba9658312ae024cbd', 'request_id': 'req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.892054 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328ba0528de244fba9658312ae024cbd', 'request_id': 'req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.900403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328ba0528de244fba9658312ae024cbd', 'request_id': 'req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.914629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] The token cache is being invalidate because project 328ba0528de244fba9658312ae024cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:03.914915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.915608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e241b68-e1b0-4bbb-89c4-2b9ac11925c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:03.917608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1377/2750] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:03 2022] DELETE /identity/v3/projects/328ba0528de244fba9658312ae024cbd => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:45:03.922851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2eea1f-b438-420b-bc70-42f2191ef1fd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.924100 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2eea1f-b438-420b-bc70-42f2191ef1fd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.924249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2eea1f-b438-420b-bc70-42f2191ef1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.950011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ce2eea1f-b438-420b-bc70-42f2191ef1fd None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:03.951806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ce2eea1f-b438-420b-bc70-42f2191ef1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:04.024974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1374/2751] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:04.034155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b8e50-a4f4-40c7-a015-85e831dd1b1e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.034428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b8e50-a4f4-40c7-a015-85e831dd1b1e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.034828 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b8e50-a4f4-40c7-a015-85e831dd1b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.064127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e66b8e50-a4f4-40c7-a015-85e831dd1b1e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:04.067028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e66b8e50-a4f4-40c7-a015-85e831dd1b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:04.081335 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1378/2752] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:04.088308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2eea1f-b438-420b-bc70-42f2191ef1fd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.110507 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2eea1f-b438-420b-bc70-42f2191ef1fd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.119068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6be3a5-d3f9-4128-a318-e15c610295c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.120326 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6be3a5-d3f9-4128-a318-e15c610295c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.121063 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6be3a5-d3f9-4128-a318-e15c610295c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.121670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6be3a5-d3f9-4128-a318-e15c610295c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.122882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6be3a5-d3f9-4128-a318-e15c610295c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:04.123409 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6be3a5-d3f9-4128-a318-e15c610295c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.139357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6be3a5-d3f9-4128-a318-e15c610295c2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.149610 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1375/2753] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:04.156225 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b8e50-a4f4-40c7-a015-85e831dd1b1e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.172935 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b8e50-a4f4-40c7-a015-85e831dd1b1e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.180291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090599ce-fbf3-4153-b5c4-51f254dfacf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.181313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090599ce-fbf3-4153-b5c4-51f254dfacf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.181842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090599ce-fbf3-4153-b5c4-51f254dfacf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.182320 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090599ce-fbf3-4153-b5c4-51f254dfacf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.183474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090599ce-fbf3-4153-b5c4-51f254dfacf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.187836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090599ce-fbf3-4153-b5c4-51f254dfacf0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.223200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1379/2754] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:04 2022] POST /identity/v3/projects => generated 386 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:04.227614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6be3a5-d3f9-4128-a318-e15c610295c2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.237991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6be3a5-d3f9-4128-a318-e15c610295c2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.282038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:04.297983 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19003603-e23f-4231-af9b-b82b99c777b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.298679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19003603-e23f-4231-af9b-b82b99c777b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.299072 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19003603-e23f-4231-af9b-b82b99c777b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.299180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19003603-e23f-4231-af9b-b82b99c777b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.299810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19003603-e23f-4231-af9b-b82b99c777b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.301802 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19003603-e23f-4231-af9b-b82b99c777b5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.323785 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1376/2755] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:04 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:04.328333 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090599ce-fbf3-4153-b5c4-51f254dfacf0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.339601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090599ce-fbf3-4153-b5c4-51f254dfacf0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.344172 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b99f2-31c5-4181-819d-e04b8c911ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.344759 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b99f2-31c5-4181-819d-e04b8c911ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.345034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b99f2-31c5-4181-819d-e04b8c911ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.345319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b99f2-31c5-4181-819d-e04b8c911ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.345967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b99f2-31c5-4181-819d-e04b8c911ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.347608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b99f2-31c5-4181-819d-e04b8c911ebe admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.357581 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1380/2756] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:04.361841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19003603-e23f-4231-af9b-b82b99c777b5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.372067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19003603-e23f-4231-af9b-b82b99c777b5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.377266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5f7e-c7ef-421f-8aa7-d60244cdc46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.377985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5f7e-c7ef-421f-8aa7-d60244cdc46f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.378178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5f7e-c7ef-421f-8aa7-d60244cdc46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.378452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5f7e-c7ef-421f-8aa7-d60244cdc46f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.379668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5f7e-c7ef-421f-8aa7-d60244cdc46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.381278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5f7e-c7ef-421f-8aa7-d60244cdc46f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.393253 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1377/2757] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:04.398388 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b99f2-31c5-4181-819d-e04b8c911ebe admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.408544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b99f2-31c5-4181-819d-e04b8c911ebe admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.413519 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d3896-6b6e-4f6c-ab2b-6f338108f65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.414220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d3896-6b6e-4f6c-ab2b-6f338108f65e admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.414493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d3896-6b6e-4f6c-ab2b-6f338108f65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.414762 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d3896-6b6e-4f6c-ab2b-6f338108f65e admin admin] PATH_INFO: `/v3/projects/4cc8e394d15b4995ac92460376c67fb7/users/cf91df130fd244a0841a468cb16cb03b/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.430775 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d3896-6b6e-4f6c-ab2b-6f338108f65e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc8e394d15b4995ac92460376c67fb7, user_id=cf91df130fd244a0841a468cb16cb03b, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.433374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d3896-6b6e-4f6c-ab2b-6f338108f65e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.446755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1381/2758] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:04 2022] PUT /identity/v3/projects/4cc8e394d15b4995ac92460376c67fb7/users/cf91df130fd244a0841a468cb16cb03b/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:04.457771 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413486fc-28f8-449e-865d-4b118472dd29 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.458062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413486fc-28f8-449e-865d-4b118472dd29 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.458338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413486fc-28f8-449e-865d-4b118472dd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.485534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-413486fc-28f8-449e-865d-4b118472dd29 None None] MFA Rules not processed for user `cf91df130fd244a0841a468cb16cb03b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:04.486781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-413486fc-28f8-449e-865d-4b118472dd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:04.552817 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1378/2759] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:04.564607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d3896-6b6e-4f6c-ab2b-6f338108f65e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.566418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c9d3896-6b6e-4f6c-ab2b-6f338108f65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:04.624417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d3896-6b6e-4f6c-ab2b-6f338108f65e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.628042 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497a602-48d7-42c5-8078-1a1853353b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.628643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497a602-48d7-42c5-8078-1a1853353b55 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.629546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497a602-48d7-42c5-8078-1a1853353b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.629546 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497a602-48d7-42c5-8078-1a1853353b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.638190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497a602-48d7-42c5-8078-1a1853353b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.639630 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497a602-48d7-42c5-8078-1a1853353b55 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.648555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1382/2760] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:04.668436 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413486fc-28f8-449e-865d-4b118472dd29 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.730427 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413486fc-28f8-449e-865d-4b118472dd29 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.735053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.735679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.735960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.736231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] PATH_INFO: `/v3/users/cf91df130fd244a0841a468cb16cb03b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.737369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf91df130fd244a0841a468cb16cb03b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.739961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.785391 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf91df130fd244a0841a468cb16cb03b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:04.793225 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf91df130fd244a0841a468cb16cb03b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:04.794638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf91df130fd244a0841a468cb16cb03b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:04.795127 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:04.809157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf91df130fd244a0841a468cb16cb03b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:04.815851 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf91df130fd244a0841a468cb16cb03b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:04.822453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1379/2761] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:04 2022] DELETE /identity/v3/users/cf91df130fd244a0841a468cb16cb03b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:04.826982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbf971-61a5-4218-8f3b-c48a200451f9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.827526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbf971-61a5-4218-8f3b-c48a200451f9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.827526 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbf971-61a5-4218-8f3b-c48a200451f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.846563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a6bbf971-61a5-4218-8f3b-c48a200451f9 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:04.848301 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a6bbf971-61a5-4218-8f3b-c48a200451f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:04.905027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1383/2762] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:04.915801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.917372 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:04.975501 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c6cbd-0a3c-47c9-860c-0336ff9c5cf4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.979087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40103914-6a32-443f-888a-4578238e2760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.979689 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40103914-6a32-443f-888a-4578238e2760 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.979968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40103914-6a32-443f-888a-4578238e2760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.980238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40103914-6a32-443f-888a-4578238e2760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.990110 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40103914-6a32-443f-888a-4578238e2760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.991516 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40103914-6a32-443f-888a-4578238e2760 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.000972 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1380/2763] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:45:05.283566 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbf971-61a5-4218-8f3b-c48a200451f9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.286784 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a6bbf971-61a5-4218-8f3b-c48a200451f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:05.313179 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbf971-61a5-4218-8f3b-c48a200451f9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.320084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.320891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.321233 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.321965 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] PATH_INFO: `/v3/projects/4cc8e394d15b4995ac92460376c67fb7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.323999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cc8e394d15b4995ac92460376c67fb7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.326331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.334695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:05.376708 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40103914-6a32-443f-888a-4578238e2760 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.378286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-40103914-6a32-443f-888a-4578238e2760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:05.383008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cc8e394d15b4995ac92460376c67fb7', 'request_id': 'req-500adcc0-384a-42af-8557-0278f90494f4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:05.390748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cc8e394d15b4995ac92460376c67fb7', 'request_id': 'req-500adcc0-384a-42af-8557-0278f90494f4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:05.397025 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cc8e394d15b4995ac92460376c67fb7', 'request_id': 'req-500adcc0-384a-42af-8557-0278f90494f4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:05.412070 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] The token cache is being invalidate because project 4cc8e394d15b4995ac92460376c67fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:05.412365 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:05.413007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-500adcc0-384a-42af-8557-0278f90494f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:05.414690 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1384/2764] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:05 2022] DELETE /identity/v3/projects/4cc8e394d15b4995ac92460376c67fb7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:05.419498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f6bd-dca0-4736-b0d2-98329d912c3d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.419778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f6bd-dca0-4736-b0d2-98329d912c3d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.420050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f6bd-dca0-4736-b0d2-98329d912c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.438801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40103914-6a32-443f-888a-4578238e2760 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.440105 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f8e9f6bd-dca0-4736-b0d2-98329d912c3d None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:05.441469 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f8e9f6bd-dca0-4736-b0d2-98329d912c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:05.491876 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910f587-895d-4f20-a684-ae6ab88cc623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.492482 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910f587-895d-4f20-a684-ae6ab88cc623 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.492766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910f587-895d-4f20-a684-ae6ab88cc623 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.493041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910f587-895d-4f20-a684-ae6ab88cc623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.494794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b910f587-895d-4f20-a684-ae6ab88cc623 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:05.503882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1385/2765] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:45:05.512528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6c2ed-61ce-4fea-9016-7ff8f27cd3c2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.512945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6c2ed-61ce-4fea-9016-7ff8f27cd3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.513208 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6c2ed-61ce-4fea-9016-7ff8f27cd3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.533021 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-20d6c2ed-61ce-4fea-9016-7ff8f27cd3c2 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:05.534423 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-20d6c2ed-61ce-4fea-9016-7ff8f27cd3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:05.543441 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1386/2766] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:45:05.545956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:05.547671 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6c2ed-61ce-4fea-9016-7ff8f27cd3c2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.557768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6c2ed-61ce-4fea-9016-7ff8f27cd3c2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.561960 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910f587-895d-4f20-a684-ae6ab88cc623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.562404 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc5c92-61c2-4815-a3a6-c32195943063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.563017 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc5c92-61c2-4815-a3a6-c32195943063 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.563291 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc5c92-61c2-4815-a3a6-c32195943063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.563398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910f587-895d-4f20-a684-ae6ab88cc623 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.563556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc5c92-61c2-4815-a3a6-c32195943063 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.564228 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc5c92-61c2-4815-a3a6-c32195943063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:05.564524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc5c92-61c2-4815-a3a6-c32195943063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.566159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc5c92-61c2-4815-a3a6-c32195943063 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.572738 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1381/2767] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:05 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:45:05.578161 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1387/2768] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:45:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:05.582348 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910f587-895d-4f20-a684-ae6ab88cc623 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.593226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910f587-895d-4f20-a684-ae6ab88cc623 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.597784 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfb452-d4cb-4f19-a46e-744577310a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.598930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfb452-d4cb-4f19-a46e-744577310a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.599211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfb452-d4cb-4f19-a46e-744577310a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.599709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfb452-d4cb-4f19-a46e-744577310a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.600480 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfb452-d4cb-4f19-a46e-744577310a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.602378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfb452-d4cb-4f19-a46e-744577310a14 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.631451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1382/2769] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:05 2022] POST /identity/v3/projects => generated 376 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:45:05.635902 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc5c92-61c2-4815-a3a6-c32195943063 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.646701 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc5c92-61c2-4815-a3a6-c32195943063 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.711159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e85730-1e1b-4998-9a99-0ac43fc4d8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.712046 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e85730-1e1b-4998-9a99-0ac43fc4d8de admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.712326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e85730-1e1b-4998-9a99-0ac43fc4d8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.712607 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e85730-1e1b-4998-9a99-0ac43fc4d8de admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.713248 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e85730-1e1b-4998-9a99-0ac43fc4d8de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.715201 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e85730-1e1b-4998-9a99-0ac43fc4d8de admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.740237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1388/2770] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:05 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:45:05.744181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfb452-d4cb-4f19-a46e-744577310a14 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.755889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfb452-d4cb-4f19-a46e-744577310a14 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.761474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8973c2-e9f5-4a52-90b4-b25e4b4a5280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.762325 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8973c2-e9f5-4a52-90b4-b25e4b4a5280 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.762674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8973c2-e9f5-4a52-90b4-b25e4b4a5280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.763014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8973c2-e9f5-4a52-90b4-b25e4b4a5280 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.763872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8973c2-e9f5-4a52-90b4-b25e4b4a5280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.765535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8973c2-e9f5-4a52-90b4-b25e4b4a5280 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.777933 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1383/2771] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:05 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:45:05.782953 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e85730-1e1b-4998-9a99-0ac43fc4d8de admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.793391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:05.798209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e85730-1e1b-4998-9a99-0ac43fc4d8de admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.806800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb2b3d-67f9-49a9-a360-0ac40ef0e077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.807616 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2b3d-67f9-49a9-a360-0ac40ef0e077 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.807980 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2b3d-67f9-49a9-a360-0ac40ef0e077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.808322 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2b3d-67f9-49a9-a360-0ac40ef0e077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.809796 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb2b3d-67f9-49a9-a360-0ac40ef0e077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.811544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb2b3d-67f9-49a9-a360-0ac40ef0e077 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.823704 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1389/2772] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:05 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:05.828235 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8973c2-e9f5-4a52-90b4-b25e4b4a5280 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.840038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8973c2-e9f5-4a52-90b4-b25e4b4a5280 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.845624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0960f6b-1054-45dc-9ffd-8fa6b3ddc34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.846278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0960f6b-1054-45dc-9ffd-8fa6b3ddc34e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.846622 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0960f6b-1054-45dc-9ffd-8fa6b3ddc34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.846820 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0960f6b-1054-45dc-9ffd-8fa6b3ddc34e admin admin] PATH_INFO: `/v3/projects/bbcec45087c04b2283220537e4a51dbd/users/ebdcd179b56a4cdc8a33f12581fcc795/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.865066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0960f6b-1054-45dc-9ffd-8fa6b3ddc34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbcec45087c04b2283220537e4a51dbd, user_id=ebdcd179b56a4cdc8a33f12581fcc795, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.867818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0960f6b-1054-45dc-9ffd-8fa6b3ddc34e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.881715 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1384/2773] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:05 2022] PUT /identity/v3/projects/bbcec45087c04b2283220537e4a51dbd/users/ebdcd179b56a4cdc8a33f12581fcc795/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:45:05.951721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f4b7b-a268-4194-a352-25642508b6e8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.952277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f4b7b-a268-4194-a352-25642508b6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.952277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f4b7b-a268-4194-a352-25642508b6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.982218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-922f4b7b-a268-4194-a352-25642508b6e8 None None] MFA Rules not processed for user `ebdcd179b56a4cdc8a33f12581fcc795`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:05.983431 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-922f4b7b-a268-4194-a352-25642508b6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:06.053305 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1390/2774] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:06.068624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0960f6b-1054-45dc-9ffd-8fa6b3ddc34e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:06.070343 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a0960f6b-1054-45dc-9ffd-8fa6b3ddc34e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:06.132132 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0960f6b-1054-45dc-9ffd-8fa6b3ddc34e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:06.136104 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291115e1-7489-49ab-9efe-cb96af0882bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:06.136659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291115e1-7489-49ab-9efe-cb96af0882bf service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:06.136942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291115e1-7489-49ab-9efe-cb96af0882bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:06.137210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291115e1-7489-49ab-9efe-cb96af0882bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:06.141603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f4b7b-a268-4194-a352-25642508b6e8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:06.142972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-922f4b7b-a268-4194-a352-25642508b6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:06.146655 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291115e1-7489-49ab-9efe-cb96af0882bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:06.148074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291115e1-7489-49ab-9efe-cb96af0882bf service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:06.154905 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f4b7b-a268-4194-a352-25642508b6e8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:06.157215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1385/2775] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:06.158776 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd3ce-3ae9-410e-8971-d6de5d13543c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:06.159379 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd3ce-3ae9-410e-8971-d6de5d13543c service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:06.159658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd3ce-3ae9-410e-8971-d6de5d13543c service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:06.159925 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd3ce-3ae9-410e-8971-d6de5d13543c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:06.161763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-246dd3ce-3ae9-410e-8971-d6de5d13543c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:06.227449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd3ce-3ae9-410e-8971-d6de5d13543c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:06.228778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd3ce-3ae9-410e-8971-d6de5d13543c service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:06.238721 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1391/2776] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:06 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:11.200519 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291115e1-7489-49ab-9efe-cb96af0882bf service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:11.201980 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-291115e1-7489-49ab-9efe-cb96af0882bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:11.269932 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291115e1-7489-49ab-9efe-cb96af0882bf service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:11.275386 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:11.276012 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:11.276283 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:11.276547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] PATH_INFO: `/v3/users/830b7b41b0b149de8c3fb3b33ef56b94` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:11.277699 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=830b7b41b0b149de8c3fb3b33ef56b94)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:11.279597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:11.324173 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:11.339143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830b7b41b0b149de8c3fb3b33ef56b94'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.349333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830b7b41b0b149de8c3fb3b33ef56b94'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.349333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830b7b41b0b149de8c3fb3b33ef56b94'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.364798 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830b7b41b0b149de8c3fb3b33ef56b94'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.371985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830b7b41b0b149de8c3fb3b33ef56b94'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.379275 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1386/2777] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:11 2022] DELETE /identity/v3/users/830b7b41b0b149de8c3fb3b33ef56b94 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:11.384370 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7c2f3-787f-49c5-91be-98efdbac732b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:11.384971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7c2f3-787f-49c5-91be-98efdbac732b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:11.384971 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7c2f3-787f-49c5-91be-98efdbac732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:11.404349 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f7f7c2f3-787f-49c5-91be-98efdbac732b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:11.405545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f7f7c2f3-787f-49c5-91be-98efdbac732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:11.465028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1392/2778] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:11.475498 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:11.476900 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:11.534389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1005ad-d650-44ff-885f-799019c20324 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:11.538664 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab989acd-5a9b-421a-91d9-4b0096fb3d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:11.539256 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab989acd-5a9b-421a-91d9-4b0096fb3d51 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:11.539535 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab989acd-5a9b-421a-91d9-4b0096fb3d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:11.539794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab989acd-5a9b-421a-91d9-4b0096fb3d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:11.549067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab989acd-5a9b-421a-91d9-4b0096fb3d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:11.550506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab989acd-5a9b-421a-91d9-4b0096fb3d51 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:11.559905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1387/2779] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:11.842657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7c2f3-787f-49c5-91be-98efdbac732b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:11.844075 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f7f7c2f3-787f-49c5-91be-98efdbac732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:11.857723 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7c2f3-787f-49c5-91be-98efdbac732b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:11.862409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:11.863044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:11.863341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:11.863590 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] PATH_INFO: `/v3/projects/0a40a06f36814d91a69f580ab398d3c7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:11.865355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a40a06f36814d91a69f580ab398d3c7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:11.867290 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:11.880019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:11.918037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a40a06f36814d91a69f580ab398d3c7', 'request_id': 'req-505e14df-5db5-4783-82df-5205a5213db0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.926028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a40a06f36814d91a69f580ab398d3c7', 'request_id': 'req-505e14df-5db5-4783-82df-5205a5213db0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.932561 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a40a06f36814d91a69f580ab398d3c7', 'request_id': 'req-505e14df-5db5-4783-82df-5205a5213db0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.944051 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] The token cache is being invalidate because project 0a40a06f36814d91a69f580ab398d3c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:11.944351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.945052 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:11.946836 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1393/2780] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:11 2022] DELETE /identity/v3/projects/0a40a06f36814d91a69f580ab398d3c7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:11.950875 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab989acd-5a9b-421a-91d9-4b0096fb3d51 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:11.961364 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab989acd-5a9b-421a-91d9-4b0096fb3d51 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:12.021758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:12.022393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.022674 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.022947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] PATH_INFO: `/v3/users/4318aeb27c124372aeb3e35f982af736` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.024113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4318aeb27c124372aeb3e35f982af736)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:12.026053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:12.081757 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4318aeb27c124372aeb3e35f982af736'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.088902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4318aeb27c124372aeb3e35f982af736'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.089517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4318aeb27c124372aeb3e35f982af736'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.090728 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:12.104263 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4318aeb27c124372aeb3e35f982af736'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.111066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4318aeb27c124372aeb3e35f982af736'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.117437 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1388/2781] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:11 2022] DELETE /identity/v3/users/4318aeb27c124372aeb3e35f982af736 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:45:12.395664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:12.397373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:12.412572 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e14df-5db5-4783-82df-5205a5213db0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:12.470538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f01153-66bf-4501-a235-352380c380db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:12.471078 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f01153-66bf-4501-a235-352380c380db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.471351 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f01153-66bf-4501-a235-352380c380db admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.471617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f01153-66bf-4501-a235-352380c380db admin admin] PATH_INFO: `/v3/projects/99c96206d9994dc89ed2f79fc8ac4cab` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.472713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f01153-66bf-4501-a235-352380c380db admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c96206d9994dc89ed2f79fc8ac4cab)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:12.475148 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f01153-66bf-4501-a235-352380c380db admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:12.521461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61f01153-66bf-4501-a235-352380c380db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c96206d9994dc89ed2f79fc8ac4cab', 'request_id': 'req-61f01153-66bf-4501-a235-352380c380db'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.529230 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61f01153-66bf-4501-a235-352380c380db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c96206d9994dc89ed2f79fc8ac4cab', 'request_id': 'req-61f01153-66bf-4501-a235-352380c380db'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.536434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61f01153-66bf-4501-a235-352380c380db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c96206d9994dc89ed2f79fc8ac4cab', 'request_id': 'req-61f01153-66bf-4501-a235-352380c380db'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.548717 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61f01153-66bf-4501-a235-352380c380db admin admin] The token cache is being invalidate because project 99c96206d9994dc89ed2f79fc8ac4cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:12.549006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61f01153-66bf-4501-a235-352380c380db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.549693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-61f01153-66bf-4501-a235-352380c380db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.551188 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1394/2782] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:12 2022] DELETE /identity/v3/projects/99c96206d9994dc89ed2f79fc8ac4cab => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:12.555852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:12.557298 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:12.570254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71cbba-97e9-4b3a-8382-f46bbd906df5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:12.627478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:12.628107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.628388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.628662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] PATH_INFO: `/v3/users/8cdca3c796f8495cb1f5861f59ec1365` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.629866 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cdca3c796f8495cb1f5861f59ec1365)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:12.631973 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:12.681796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cdca3c796f8495cb1f5861f59ec1365'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.689308 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cdca3c796f8495cb1f5861f59ec1365'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.690026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cdca3c796f8495cb1f5861f59ec1365'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.702096 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cdca3c796f8495cb1f5861f59ec1365'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.709556 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-082fd6b7-d09a-465c-9b46-16da1f34f0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cdca3c796f8495cb1f5861f59ec1365'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:12.715424 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1389/2783] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:12 2022] DELETE /identity/v3/users/8cdca3c796f8495cb1f5861f59ec1365 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:12.991658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f01153-66bf-4501-a235-352380c380db admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:12.993080 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-61f01153-66bf-4501-a235-352380c380db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:13.007415 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f01153-66bf-4501-a235-352380c380db admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.040286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.070521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.071150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.071447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.071713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] PATH_INFO: `/v3/projects/ca2fecebb657455b80d83fb2a6638d9b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.072832 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2fecebb657455b80d83fb2a6638d9b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.074763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.123474 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2fecebb657455b80d83fb2a6638d9b', 'request_id': 'req-440f66b6-949d-428e-acbb-3fd354853414'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:13.131050 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2fecebb657455b80d83fb2a6638d9b', 'request_id': 'req-440f66b6-949d-428e-acbb-3fd354853414'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:13.136844 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2fecebb657455b80d83fb2a6638d9b', 'request_id': 'req-440f66b6-949d-428e-acbb-3fd354853414'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:13.148358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] The token cache is being invalidate because project ca2fecebb657455b80d83fb2a6638d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:13.148659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:13.149418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-440f66b6-949d-428e-acbb-3fd354853414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:13.150934 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1395/2784] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:12 2022] DELETE /identity/v3/projects/ca2fecebb657455b80d83fb2a6638d9b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:13.156969 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d5ac4-fc37-4c10-b592-f3f9b5d0c88e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.157258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d5ac4-fc37-4c10-b592-f3f9b5d0c88e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.157534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d5ac4-fc37-4c10-b592-f3f9b5d0c88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.177594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-097d5ac4-fc37-4c10-b592-f3f9b5d0c88e None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:13.178821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-097d5ac4-fc37-4c10-b592-f3f9b5d0c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:13.224903 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.240892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1390/2785] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:45:13.248765 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2f8ed-61e4-4ba5-80a9-e7b0b2649a57 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.249082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2f8ed-61e4-4ba5-80a9-e7b0b2649a57 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.249353 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2f8ed-61e4-4ba5-80a9-e7b0b2649a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.269475 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-49a2f8ed-61e4-4ba5-80a9-e7b0b2649a57 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:13.271283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-49a2f8ed-61e4-4ba5-80a9-e7b0b2649a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:13.279171 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1396/2786] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:13.283394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d5ac4-fc37-4c10-b592-f3f9b5d0c88e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.293286 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d5ac4-fc37-4c10-b592-f3f9b5d0c88e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.297864 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6ee06-76ea-455f-a18c-4912b7f9f52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.298455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6ee06-76ea-455f-a18c-4912b7f9f52f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.298729 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6ee06-76ea-455f-a18c-4912b7f9f52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.298995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6ee06-76ea-455f-a18c-4912b7f9f52f admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.299626 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6ee06-76ea-455f-a18c-4912b7f9f52f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:13.299922 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6ee06-76ea-455f-a18c-4912b7f9f52f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.301499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6ee06-76ea-455f-a18c-4912b7f9f52f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.314006 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1391/2787] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:13.319545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2f8ed-61e4-4ba5-80a9-e7b0b2649a57 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.328873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2f8ed-61e4-4ba5-80a9-e7b0b2649a57 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.334991 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0767dba-ebc9-4294-889d-a4e5bc6282b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.335589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767dba-ebc9-4294-889d-a4e5bc6282b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.335864 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767dba-ebc9-4294-889d-a4e5bc6282b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.336133 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767dba-ebc9-4294-889d-a4e5bc6282b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.336760 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0767dba-ebc9-4294-889d-a4e5bc6282b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.338673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0767dba-ebc9-4294-889d-a4e5bc6282b4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.367243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1397/2788] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:13 2022] POST /identity/v3/projects => generated 400 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:13.371499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6ee06-76ea-455f-a18c-4912b7f9f52f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.384340 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6ee06-76ea-455f-a18c-4912b7f9f52f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.441849 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a68d1e-2c2a-408a-a427-acde0bc27403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.443037 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a68d1e-2c2a-408a-a427-acde0bc27403 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.443320 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a68d1e-2c2a-408a-a427-acde0bc27403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.443596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a68d1e-2c2a-408a-a427-acde0bc27403 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.444233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a68d1e-2c2a-408a-a427-acde0bc27403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.446212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a68d1e-2c2a-408a-a427-acde0bc27403 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.467210 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1392/2789] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:13 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:13.471381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0767dba-ebc9-4294-889d-a4e5bc6282b4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.481889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0767dba-ebc9-4294-889d-a4e5bc6282b4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.486471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec904ae-8324-4b50-b7e6-06e3f310b696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.487068 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec904ae-8324-4b50-b7e6-06e3f310b696 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.487347 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec904ae-8324-4b50-b7e6-06e3f310b696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.487617 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec904ae-8324-4b50-b7e6-06e3f310b696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.488284 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec904ae-8324-4b50-b7e6-06e3f310b696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.490437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec904ae-8324-4b50-b7e6-06e3f310b696 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.500491 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1398/2790] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:13.504414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a68d1e-2c2a-408a-a427-acde0bc27403 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.515440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a68d1e-2c2a-408a-a427-acde0bc27403 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.520174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf9972-940c-49da-b7ee-2f7d71d54705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.521060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9972-940c-49da-b7ee-2f7d71d54705 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.521347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9972-940c-49da-b7ee-2f7d71d54705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.521649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9972-940c-49da-b7ee-2f7d71d54705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.522290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf9972-940c-49da-b7ee-2f7d71d54705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.523874 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf9972-940c-49da-b7ee-2f7d71d54705 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.534679 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1393/2791] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:13.539119 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec904ae-8324-4b50-b7e6-06e3f310b696 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.549480 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec904ae-8324-4b50-b7e6-06e3f310b696 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.554781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50648ad1-62a9-462b-b7bb-7c6e2d800b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.555425 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50648ad1-62a9-462b-b7bb-7c6e2d800b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.555700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50648ad1-62a9-462b-b7bb-7c6e2d800b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.555966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50648ad1-62a9-462b-b7bb-7c6e2d800b33 admin admin] PATH_INFO: `/v3/projects/2022359251be4b55b362d830314680d0/users/dd8a9103c3b7472f9daac8988ffb5bf8/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.571001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50648ad1-62a9-462b-b7bb-7c6e2d800b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2022359251be4b55b362d830314680d0, user_id=dd8a9103c3b7472f9daac8988ffb5bf8, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.573652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50648ad1-62a9-462b-b7bb-7c6e2d800b33 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.586703 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1399/2792] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:13 2022] PUT /identity/v3/projects/2022359251be4b55b362d830314680d0/users/dd8a9103c3b7472f9daac8988ffb5bf8/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:45:13.598452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281ac4c-cde1-4764-87c6-4ec97eacaa0b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.598944 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281ac4c-cde1-4764-87c6-4ec97eacaa0b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.599087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281ac4c-cde1-4764-87c6-4ec97eacaa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.626956 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e281ac4c-cde1-4764-87c6-4ec97eacaa0b None None] MFA Rules not processed for user `dd8a9103c3b7472f9daac8988ffb5bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:13.628163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e281ac4c-cde1-4764-87c6-4ec97eacaa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:13.696763 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1394/2793] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:13 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:13.708928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50648ad1-62a9-462b-b7bb-7c6e2d800b33 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.710657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-50648ad1-62a9-462b-b7bb-7c6e2d800b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:13.771928 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50648ad1-62a9-462b-b7bb-7c6e2d800b33 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.775516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb93880-b67d-480a-8560-f961365cfe95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.776121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb93880-b67d-480a-8560-f961365cfe95 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.776402 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb93880-b67d-480a-8560-f961365cfe95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.776678 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb93880-b67d-480a-8560-f961365cfe95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.784933 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb93880-b67d-480a-8560-f961365cfe95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.787253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb93880-b67d-480a-8560-f961365cfe95 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.791895 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.797970 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1400/2794] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:22.941589 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281ac4c-cde1-4764-87c6-4ec97eacaa0b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.992961 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:23.011274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281ac4c-cde1-4764-87c6-4ec97eacaa0b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.015896 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.016528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.016808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.017085 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] PATH_INFO: `/v3/users/dd8a9103c3b7472f9daac8988ffb5bf8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.018227 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd8a9103c3b7472f9daac8988ffb5bf8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.020174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.067374 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8a9103c3b7472f9daac8988ffb5bf8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.075014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8a9103c3b7472f9daac8988ffb5bf8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.075838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8a9103c3b7472f9daac8988ffb5bf8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.087640 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8a9103c3b7472f9daac8988ffb5bf8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.094234 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8a9103c3b7472f9daac8988ffb5bf8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.100580 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1395/2795] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:22 2022] DELETE /identity/v3/users/dd8a9103c3b7472f9daac8988ffb5bf8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:23.106004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e623487-9bd2-43c1-8b2a-d77768600391 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.106383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e623487-9bd2-43c1-8b2a-d77768600391 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.106520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e623487-9bd2-43c1-8b2a-d77768600391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.128237 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3e623487-9bd2-43c1-8b2a-d77768600391 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:23.129643 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3e623487-9bd2-43c1-8b2a-d77768600391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:23.189629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1401/2796] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:23.199830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.201226 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:23.259862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224dcf2-3c9b-4fc4-8761-bd074c96e133 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.263539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9479f43-86e6-47aa-97ae-d2fda22ff2b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.264129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9479f43-86e6-47aa-97ae-d2fda22ff2b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.264404 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9479f43-86e6-47aa-97ae-d2fda22ff2b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.264672 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9479f43-86e6-47aa-97ae-d2fda22ff2b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.274819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9479f43-86e6-47aa-97ae-d2fda22ff2b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.276231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9479f43-86e6-47aa-97ae-d2fda22ff2b7 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.286831 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1396/2797] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:23.562809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e623487-9bd2-43c1-8b2a-d77768600391 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.564331 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3e623487-9bd2-43c1-8b2a-d77768600391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:23.577014 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e623487-9bd2-43c1-8b2a-d77768600391 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.581951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.583209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.583545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.583814 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] PATH_INFO: `/v3/projects/2022359251be4b55b362d830314680d0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.584943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2022359251be4b55b362d830314680d0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.586886 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.635664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2022359251be4b55b362d830314680d0', 'request_id': 'req-021e5722-57db-46e2-a79d-9ce6d0785e50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.643300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2022359251be4b55b362d830314680d0', 'request_id': 'req-021e5722-57db-46e2-a79d-9ce6d0785e50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.650236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2022359251be4b55b362d830314680d0', 'request_id': 'req-021e5722-57db-46e2-a79d-9ce6d0785e50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.662477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] The token cache is being invalidate because project 2022359251be4b55b362d830314680d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:23.662851 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.663574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-021e5722-57db-46e2-a79d-9ce6d0785e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:23.665139 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1402/2798] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:23 2022] DELETE /identity/v3/projects/2022359251be4b55b362d830314680d0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:29.099654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9479f43-86e6-47aa-97ae-d2fda22ff2b7 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.101077 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e9479f43-86e6-47aa-97ae-d2fda22ff2b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:29.114254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9479f43-86e6-47aa-97ae-d2fda22ff2b7 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.170982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.171596 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.171871 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.172137 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] PATH_INFO: `/v3/users/ebdcd179b56a4cdc8a33f12581fcc795` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.173247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdcd179b56a4cdc8a33f12581fcc795)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.175181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.185426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.224191 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdcd179b56a4cdc8a33f12581fcc795'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.231632 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdcd179b56a4cdc8a33f12581fcc795'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.232378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdcd179b56a4cdc8a33f12581fcc795'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.244877 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdcd179b56a4cdc8a33f12581fcc795'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.251838 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdcd179b56a4cdc8a33f12581fcc795'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.257758 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1397/2799] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:45:29 2022] DELETE /identity/v3/users/ebdcd179b56a4cdc8a33f12581fcc795 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:29.262299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6cdab-fe31-406d-bc33-9ffe878eaa98 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.262800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6cdab-fe31-406d-bc33-9ffe878eaa98 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.262922 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6cdab-fe31-406d-bc33-9ffe878eaa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.282376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-34f6cdab-fe31-406d-bc33-9ffe878eaa98 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:29.283699 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34f6cdab-fe31-406d-bc33-9ffe878eaa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:29.343593 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1403/2800] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:29.354066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.355459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:29.414455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6009ac-42c8-4c72-800e-d4ba65bf01bb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.418295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65987a51-15f6-4a2c-a45d-7e7cc69407a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.418889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65987a51-15f6-4a2c-a45d-7e7cc69407a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.419163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65987a51-15f6-4a2c-a45d-7e7cc69407a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.419431 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65987a51-15f6-4a2c-a45d-7e7cc69407a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.429088 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65987a51-15f6-4a2c-a45d-7e7cc69407a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.430509 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65987a51-15f6-4a2c-a45d-7e7cc69407a5 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.439863 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1398/2801] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:29.716856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6cdab-fe31-406d-bc33-9ffe878eaa98 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.718539 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34f6cdab-fe31-406d-bc33-9ffe878eaa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:29.735556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6cdab-fe31-406d-bc33-9ffe878eaa98 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.741348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.741915 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.742190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.742494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] PATH_INFO: `/v3/projects/bbcec45087c04b2283220537e4a51dbd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.743615 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbcec45087c04b2283220537e4a51dbd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.745998 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.767473 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.794055 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbcec45087c04b2283220537e4a51dbd', 'request_id': 'req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.802719 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbcec45087c04b2283220537e4a51dbd', 'request_id': 'req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.808385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbcec45087c04b2283220537e4a51dbd', 'request_id': 'req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.819517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] The token cache is being invalidate because project bbcec45087c04b2283220537e4a51dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:45:29.819809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.820461 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c5af280a-f52e-49dc-8b8b-e27c92fb0ade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:45:29.821975 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1404/2802] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:45:29 2022] DELETE /identity/v3/projects/bbcec45087c04b2283220537e4a51dbd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:29.826852 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce385b02-53eb-4f3c-9671-ab1094fe8ec1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.827139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce385b02-53eb-4f3c-9671-ab1094fe8ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.827411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce385b02-53eb-4f3c-9671-ab1094fe8ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.846937 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ce385b02-53eb-4f3c-9671-ab1094fe8ec1 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:29.848781 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ce385b02-53eb-4f3c-9671-ab1094fe8ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:29.908633 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1399/2803] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:29.917941 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8987f0-8334-4a9e-81a0-1393c97e7e04 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.918240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8987f0-8334-4a9e-81a0-1393c97e7e04 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.918545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8987f0-8334-4a9e-81a0-1393c97e7e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.939003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7b8987f0-8334-4a9e-81a0-1393c97e7e04 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:29.940199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7b8987f0-8334-4a9e-81a0-1393c97e7e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:29.948165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1405/2804] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:45:29.952767 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce385b02-53eb-4f3c-9671-ab1094fe8ec1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.963074 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce385b02-53eb-4f3c-9671-ab1094fe8ec1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.967644 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc3f5e-c0fa-4fac-8ae3-c70df50f9ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.968231 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc3f5e-c0fa-4fac-8ae3-c70df50f9ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.968504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc3f5e-c0fa-4fac-8ae3-c70df50f9ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.968769 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc3f5e-c0fa-4fac-8ae3-c70df50f9ab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.969403 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc3f5e-c0fa-4fac-8ae3-c70df50f9ab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:29.969736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc3f5e-c0fa-4fac-8ae3-c70df50f9ab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.971309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc3f5e-c0fa-4fac-8ae3-c70df50f9ab6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.983081 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1400/2805] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:29.987385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8987f0-8334-4a9e-81a0-1393c97e7e04 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.997004 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8987f0-8334-4a9e-81a0-1393c97e7e04 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.001700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5ba30-615d-47bd-b8ab-2443ba5067f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.002292 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ba30-615d-47bd-b8ab-2443ba5067f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.002575 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ba30-615d-47bd-b8ab-2443ba5067f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.002845 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ba30-615d-47bd-b8ab-2443ba5067f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.003471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5ba30-615d-47bd-b8ab-2443ba5067f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.005608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5ba30-615d-47bd-b8ab-2443ba5067f7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.032555 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1406/2806] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:29 2022] POST /identity/v3/projects => generated 380 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:30.037380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc3f5e-c0fa-4fac-8ae3-c70df50f9ab6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.047914 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc3f5e-c0fa-4fac-8ae3-c70df50f9ab6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.059795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:30.105787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c33aa1-b4fa-45c7-b3a3-1ba86b966f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.106455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c33aa1-b4fa-45c7-b3a3-1ba86b966f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.106659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c33aa1-b4fa-45c7-b3a3-1ba86b966f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.106931 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c33aa1-b4fa-45c7-b3a3-1ba86b966f0f admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.107565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c33aa1-b4fa-45c7-b3a3-1ba86b966f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.109506 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c33aa1-b4fa-45c7-b3a3-1ba86b966f0f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.131770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1401/2807] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:30 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:30.135893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5ba30-615d-47bd-b8ab-2443ba5067f7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.146856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5ba30-615d-47bd-b8ab-2443ba5067f7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.151440 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb388b1-ac74-46b0-a009-e941712d2262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.152037 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb388b1-ac74-46b0-a009-e941712d2262 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.152314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb388b1-ac74-46b0-a009-e941712d2262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.152594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb388b1-ac74-46b0-a009-e941712d2262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.153224 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb388b1-ac74-46b0-a009-e941712d2262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.154850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb388b1-ac74-46b0-a009-e941712d2262 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.165865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1407/2808] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:30.170054 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c33aa1-b4fa-45c7-b3a3-1ba86b966f0f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.180171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c33aa1-b4fa-45c7-b3a3-1ba86b966f0f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.184803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4fee7-4e97-4a99-86b0-ed6ea563c8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.185390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4fee7-4e97-4a99-86b0-ed6ea563c8c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.185709 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4fee7-4e97-4a99-86b0-ed6ea563c8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.185982 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4fee7-4e97-4a99-86b0-ed6ea563c8c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.186608 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4fee7-4e97-4a99-86b0-ed6ea563c8c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.188390 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4fee7-4e97-4a99-86b0-ed6ea563c8c9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.199195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1402/2809] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:30.203277 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb388b1-ac74-46b0-a009-e941712d2262 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.214449 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb388b1-ac74-46b0-a009-e941712d2262 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.219569 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f01e1-ed28-4383-93bb-f9f4c72a6d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.220236 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f01e1-ed28-4383-93bb-f9f4c72a6d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.220513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f01e1-ed28-4383-93bb-f9f4c72a6d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.220778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f01e1-ed28-4383-93bb-f9f4c72a6d90 admin admin] PATH_INFO: `/v3/projects/2e0189e8a6ef4f2c930f0864100a9d82/users/cf0dc05b996e4aa3a1d7605028c64f30/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.236646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f01e1-ed28-4383-93bb-f9f4c72a6d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0189e8a6ef4f2c930f0864100a9d82, user_id=cf0dc05b996e4aa3a1d7605028c64f30, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.239507 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f01e1-ed28-4383-93bb-f9f4c72a6d90 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.252332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1408/2810] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:30 2022] PUT /identity/v3/projects/2e0189e8a6ef4f2c930f0864100a9d82/users/cf0dc05b996e4aa3a1d7605028c64f30/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:30.257621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb825-e580-4932-979f-d86637991f7f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.257917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb825-e580-4932-979f-d86637991f7f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.258252 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb825-e580-4932-979f-d86637991f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.277474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8acdb825-e580-4932-979f-d86637991f7f None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:30.278722 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8acdb825-e580-4932-979f-d86637991f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:30.345493 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1403/2811] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:30.355608 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f01e1-ed28-4383-93bb-f9f4c72a6d90 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.357482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2f6f01e1-ed28-4383-93bb-f9f4c72a6d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:30.415243 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f01e1-ed28-4383-93bb-f9f4c72a6d90 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.419217 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bcf09-d7a4-49e5-9992-f96cd9785759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.419809 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bcf09-d7a4-49e5-9992-f96cd9785759 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.420092 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bcf09-d7a4-49e5-9992-f96cd9785759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.420354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bcf09-d7a4-49e5-9992-f96cd9785759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.429742 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bcf09-d7a4-49e5-9992-f96cd9785759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.431155 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bcf09-d7a4-49e5-9992-f96cd9785759 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.440337 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1409/2812] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:34.477923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0a92f-9419-4cf3-a7df-cc68aaf409a2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.478833 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0a92f-9419-4cf3-a7df-cc68aaf409a2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.479113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0a92f-9419-4cf3-a7df-cc68aaf409a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.521193 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-05a0a92f-9419-4cf3-a7df-cc68aaf409a2 None None] MFA Rules not processed for user `cf0dc05b996e4aa3a1d7605028c64f30`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:34.522985 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-05a0a92f-9419-4cf3-a7df-cc68aaf409a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:34.599858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1404/2813] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:45:34.604455 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bcf09-d7a4-49e5-9992-f96cd9785759 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.615280 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bcf09-d7a4-49e5-9992-f96cd9785759 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.620040 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd9a3f-eb9a-4e09-a4d5-52f71a14f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.620657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd9a3f-eb9a-4e09-a4d5-52f71a14f860 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.620943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd9a3f-eb9a-4e09-a4d5-52f71a14f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.621222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd9a3f-eb9a-4e09-a4d5-52f71a14f860 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.621885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd9a3f-eb9a-4e09-a4d5-52f71a14f860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.623792 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd9a3f-eb9a-4e09-a4d5-52f71a14f860 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.654124 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1410/2814] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:45:34 2022] POST /identity/v3/projects => generated 380 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:45:34.658464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0a92f-9419-4cf3-a7df-cc68aaf409a2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.670661 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0a92f-9419-4cf3-a7df-cc68aaf409a2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.697953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:34.735382 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbafc7-a1c1-47ed-b82b-41a0685c44c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.736024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbafc7-a1c1-47ed-b82b-41a0685c44c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.736305 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbafc7-a1c1-47ed-b82b-41a0685c44c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.736576 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbafc7-a1c1-47ed-b82b-41a0685c44c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.737216 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbafc7-a1c1-47ed-b82b-41a0685c44c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.739460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbafc7-a1c1-47ed-b82b-41a0685c44c9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.763271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1405/2815] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:45:34 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:45:34.767941 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd9a3f-eb9a-4e09-a4d5-52f71a14f860 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.779086 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd9a3f-eb9a-4e09-a4d5-52f71a14f860 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.784278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a82dd-4571-43db-badf-698d3163e5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.784899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a82dd-4571-43db-badf-698d3163e5ff admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.785920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a82dd-4571-43db-badf-698d3163e5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.786238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a82dd-4571-43db-badf-698d3163e5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.786856 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a82dd-4571-43db-badf-698d3163e5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.788524 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a82dd-4571-43db-badf-698d3163e5ff admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.800358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1411/2816] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:34.804800 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbafc7-a1c1-47ed-b82b-41a0685c44c9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.815572 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbafc7-a1c1-47ed-b82b-41a0685c44c9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.821184 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b5737-51e2-4741-a77f-06e239e9b548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.821889 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b5737-51e2-4741-a77f-06e239e9b548 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.822169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b5737-51e2-4741-a77f-06e239e9b548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.822447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b5737-51e2-4741-a77f-06e239e9b548 admin admin] PATH_INFO: `/v3/projects/80e13821282c4a129d1c943b531e6721/users/8bdb8c5bcbc94fea93efb84814a5ffca/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.839624 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b5737-51e2-4741-a77f-06e239e9b548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e13821282c4a129d1c943b531e6721, user_id=8bdb8c5bcbc94fea93efb84814a5ffca, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.842452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b5737-51e2-4741-a77f-06e239e9b548 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.858801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1406/2817] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:34 2022] PUT /identity/v3/projects/80e13821282c4a129d1c943b531e6721/users/8bdb8c5bcbc94fea93efb84814a5ffca/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:34.863048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a82dd-4571-43db-badf-698d3163e5ff admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.873914 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a82dd-4571-43db-badf-698d3163e5ff admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.883761 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:34.948636 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8e7bf-d0ce-45b9-be5e-e86f7ce65b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.949259 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8e7bf-d0ce-45b9-be5e-e86f7ce65b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.949538 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8e7bf-d0ce-45b9-be5e-e86f7ce65b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.949841 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8e7bf-d0ce-45b9-be5e-e86f7ce65b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.950477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8e7bf-d0ce-45b9-be5e-e86f7ce65b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.952058 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8e7bf-d0ce-45b9-be5e-e86f7ce65b65 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.962056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1412/2818] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:45:34.966446 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b5737-51e2-4741-a77f-06e239e9b548 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.983612 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b5737-51e2-4741-a77f-06e239e9b548 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.989268 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322ba28-1188-41b6-9056-e7e80e82057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.990092 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322ba28-1188-41b6-9056-e7e80e82057c admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.990451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322ba28-1188-41b6-9056-e7e80e82057c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.990789 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322ba28-1188-41b6-9056-e7e80e82057c admin admin] PATH_INFO: `/v3/projects/80e13821282c4a129d1c943b531e6721/users/8bdb8c5bcbc94fea93efb84814a5ffca/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.992856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322ba28-1188-41b6-9056-e7e80e82057c admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e13821282c4a129d1c943b531e6721, user_id=8bdb8c5bcbc94fea93efb84814a5ffca, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.995649 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322ba28-1188-41b6-9056-e7e80e82057c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:35.007011 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1407/2819] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:45:34 2022] PUT /identity/v3/projects/80e13821282c4a129d1c943b531e6721/users/8bdb8c5bcbc94fea93efb84814a5ffca/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:39.618452 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0e576-635d-43c8-83ed-133a9832517f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.619032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0e576-635d-43c8-83ed-133a9832517f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.619032 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0e576-635d-43c8-83ed-133a9832517f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.649036 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fdc0e576-635d-43c8-83ed-133a9832517f None None] MFA Rules not processed for user `8bdb8c5bcbc94fea93efb84814a5ffca`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:39.650319 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fdc0e576-635d-43c8-83ed-133a9832517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:39.726158 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1413/2820] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:39 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:39.742630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322ba28-1188-41b6-9056-e7e80e82057c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.744947 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0322ba28-1188-41b6-9056-e7e80e82057c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:39.876211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322ba28-1188-41b6-9056-e7e80e82057c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.882719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301518-cc34-4227-8ca6-4b259eae5cf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.883549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301518-cc34-4227-8ca6-4b259eae5cf7 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.883830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301518-cc34-4227-8ca6-4b259eae5cf7 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.884984 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301518-cc34-4227-8ca6-4b259eae5cf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.959118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:39.983270 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63301518-cc34-4227-8ca6-4b259eae5cf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:39.984879 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63301518-cc34-4227-8ca6-4b259eae5cf7 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:39.994505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1408/2821] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:45:46.006501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf95c-fa52-41bf-932f-cc49904c7b01 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.007079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf95c-fa52-41bf-932f-cc49904c7b01 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.007079 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf95c-fa52-41bf-932f-cc49904c7b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.027348 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-babdf95c-fa52-41bf-932f-cc49904c7b01 None None] MFA Rules not processed for user `d8259269a28a401eb4ddbe90f128fe71`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:46.028535 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-babdf95c-fa52-41bf-932f-cc49904c7b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:46.090393 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1414/2822] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:45:49.990441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:56.081823 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=82 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:57.821968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68842d7-3a27-41d0-ae1b-18de9cad2d11 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.822475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68842d7-3a27-41d0-ae1b-18de9cad2d11 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.822598 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68842d7-3a27-41d0-ae1b-18de9cad2d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.842362 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a68842d7-3a27-41d0-ae1b-18de9cad2d11 None None] MFA Rules not processed for user `cf0dc05b996e4aa3a1d7605028c64f30`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:57.843559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a68842d7-3a27-41d0-ae1b-18de9cad2d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:57.851144 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1409/2823] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:45:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:46:07.836796 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:19.581363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf95c-fa52-41bf-932f-cc49904c7b01 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:19.637446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf95c-fa52-41bf-932f-cc49904c7b01 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:19.641028 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f768699-c089-4b3f-9c7d-e68c076c420d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:19.641655 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f768699-c089-4b3f-9c7d-e68c076c420d service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:19.641931 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f768699-c089-4b3f-9c7d-e68c076c420d service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:19.642197 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f768699-c089-4b3f-9c7d-e68c076c420d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:19.651409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f768699-c089-4b3f-9c7d-e68c076c420d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:19.652907 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f768699-c089-4b3f-9c7d-e68c076c420d service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:19.663063 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1415/2824] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:19 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:46:29.658129 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=26 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:47.607157 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68842d7-3a27-41d0-ae1b-18de9cad2d11 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:47.608569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a68842d7-3a27-41d0-ae1b-18de9cad2d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:47.666591 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68842d7-3a27-41d0-ae1b-18de9cad2d11 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:47.670549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56b6d-0dc2-4229-97ce-8d9f8605ff9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:47.671153 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56b6d-0dc2-4229-97ce-8d9f8605ff9c service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:47.671429 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56b6d-0dc2-4229-97ce-8d9f8605ff9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:47.671697 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56b6d-0dc2-4229-97ce-8d9f8605ff9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:47.673415 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d2f56b6d-0dc2-4229-97ce-8d9f8605ff9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:47.687068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56b6d-0dc2-4229-97ce-8d9f8605ff9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:47.688406 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56b6d-0dc2-4229-97ce-8d9f8605ff9c service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:47.699634 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1410/2825] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:47 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:46:55.789522 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f768699-c089-4b3f-9c7d-e68c076c420d service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:55.799187 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f768699-c089-4b3f-9c7d-e68c076c420d service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:55.804273 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:55.804858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:55.805134 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:55.805397 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:55.807266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:55.822149 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:55.823579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:55.834786 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1416/2826] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:46:55 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:46:57.694673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=26 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:01.802658 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56b6d-0dc2-4229-97ce-8d9f8605ff9c service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:01.816143 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56b6d-0dc2-4229-97ce-8d9f8605ff9c service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:01.823125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f50197-1555-4f95-b5d2-7d70b198ae1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:01.823746 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f50197-1555-4f95-b5d2-7d70b198ae1e service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:01.824025 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f50197-1555-4f95-b5d2-7d70b198ae1e service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:01.824389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f50197-1555-4f95-b5d2-7d70b198ae1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:01.826221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-70f50197-1555-4f95-b5d2-7d70b198ae1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:01.842668 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f50197-1555-4f95-b5d2-7d70b198ae1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:01.844458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f50197-1555-4f95-b5d2-7d70b198ae1e service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:01.866304 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1411/2827] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:01 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:05.829658 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:10.147863 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:10.149421 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:10.238992 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22aab7-d925-40ef-ac2f-9ff1a6930ddd service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:10.246007 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:10.246755 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.246936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.247206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] PATH_INFO: `/v3/users/d8259269a28a401eb4ddbe90f128fe71` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.248390 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8259269a28a401eb4ddbe90f128fe71)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:10.250399 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:10.311418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8259269a28a401eb4ddbe90f128fe71'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.325793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8259269a28a401eb4ddbe90f128fe71'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.334038 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8259269a28a401eb4ddbe90f128fe71'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.342955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8259269a28a401eb4ddbe90f128fe71'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.344043 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d65b2cc-442c-48ad-a98f-2a93975e2b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8259269a28a401eb4ddbe90f128fe71'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.355339 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1417/2828] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:47:10 2022] DELETE /identity/v3/users/d8259269a28a401eb4ddbe90f128fe71 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:47:10.592359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f50197-1555-4f95-b5d2-7d70b198ae1e service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:10.604092 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f50197-1555-4f95-b5d2-7d70b198ae1e service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:10.674371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:10.675204 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.675485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.675760 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] PATH_INFO: `/v3/projects/8cdeb9d9f0eb4f1c805502bb30a99ebc` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.676963 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cdeb9d9f0eb4f1c805502bb30a99ebc)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:10.679150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:10.731062 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cdeb9d9f0eb4f1c805502bb30a99ebc', 'request_id': 'req-44417fa5-bf8e-4327-904a-31c67f4022aa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.737638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cdeb9d9f0eb4f1c805502bb30a99ebc', 'request_id': 'req-44417fa5-bf8e-4327-904a-31c67f4022aa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.744463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cdeb9d9f0eb4f1c805502bb30a99ebc', 'request_id': 'req-44417fa5-bf8e-4327-904a-31c67f4022aa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.758830 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] The token cache is being invalidate because project 8cdeb9d9f0eb4f1c805502bb30a99ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:47:10.759120 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.759822 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44417fa5-bf8e-4327-904a-31c67f4022aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:10.761381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1412/2829] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:47:10 2022] DELETE /identity/v3/projects/8cdeb9d9f0eb4f1c805502bb30a99ebc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:10.767048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7dd73-38fb-4904-a7d1-ca797b252aaa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.767395 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7dd73-38fb-4904-a7d1-ca797b252aaa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.767518 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7dd73-38fb-4904-a7d1-ca797b252aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.790242 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9fa7dd73-38fb-4904-a7d1-ca797b252aaa None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:10.791497 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9fa7dd73-38fb-4904-a7d1-ca797b252aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:10.864210 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1418/2830] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:47:10.872958 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8251beb-af70-4845-8323-dd5dea6f39be None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.873240 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8251beb-af70-4845-8323-dd5dea6f39be None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.873514 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8251beb-af70-4845-8323-dd5dea6f39be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.896339 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a8251beb-af70-4845-8323-dd5dea6f39be None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:10.897665 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a8251beb-af70-4845-8323-dd5dea6f39be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:10.906138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1413/2831] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:10.910694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7dd73-38fb-4904-a7d1-ca797b252aaa None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:10.921869 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7dd73-38fb-4904-a7d1-ca797b252aaa None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:10.934673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c9cda-3cd7-468d-a582-e447a5b30908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:10.934673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c9cda-3cd7-468d-a582-e447a5b30908 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.934673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c9cda-3cd7-468d-a582-e447a5b30908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.934673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c9cda-3cd7-468d-a582-e447a5b30908 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.935247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c9cda-3cd7-468d-a582-e447a5b30908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:47:10.935247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c9cda-3cd7-468d-a582-e447a5b30908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:10.935247 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c9cda-3cd7-468d-a582-e447a5b30908 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:10.943557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1419/2832] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:47:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:10.948560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8251beb-af70-4845-8323-dd5dea6f39be None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:10.958293 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8251beb-af70-4845-8323-dd5dea6f39be None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:10.963573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97039377-9e65-4269-83f8-109a62e96329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:10.964182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039377-9e65-4269-83f8-109a62e96329 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.964460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039377-9e65-4269-83f8-109a62e96329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.964724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039377-9e65-4269-83f8-109a62e96329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.965355 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97039377-9e65-4269-83f8-109a62e96329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:10.967324 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97039377-9e65-4269-83f8-109a62e96329 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.006938 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1414/2833] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:47:10 2022] POST /identity/v3/projects => generated 382 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:11.011293 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c9cda-3cd7-468d-a582-e447a5b30908 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.022283 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c9cda-3cd7-468d-a582-e447a5b30908 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.026447 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:11.086238 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8a89a-d422-4a86-b8b1-560114ec217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.086842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8a89a-d422-4a86-b8b1-560114ec217a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.087118 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8a89a-d422-4a86-b8b1-560114ec217a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.087387 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8a89a-d422-4a86-b8b1-560114ec217a admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.088005 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8a89a-d422-4a86-b8b1-560114ec217a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.089938 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8a89a-d422-4a86-b8b1-560114ec217a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.115368 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1420/2834] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:47:11 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:47:11.120363 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97039377-9e65-4269-83f8-109a62e96329 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.132694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97039377-9e65-4269-83f8-109a62e96329 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.137869 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481e09f-44c6-4aaa-bf18-65639b78b52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.138474 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481e09f-44c6-4aaa-bf18-65639b78b52e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.138748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481e09f-44c6-4aaa-bf18-65639b78b52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.139014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481e09f-44c6-4aaa-bf18-65639b78b52e admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.139654 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481e09f-44c6-4aaa-bf18-65639b78b52e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.141221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481e09f-44c6-4aaa-bf18-65639b78b52e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.152454 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1415/2835] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:11 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:47:11.156552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8a89a-d422-4a86-b8b1-560114ec217a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.167606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8a89a-d422-4a86-b8b1-560114ec217a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.174730 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c494d-2d7c-4768-b82f-0555015cac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.175341 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c494d-2d7c-4768-b82f-0555015cac46 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.175614 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c494d-2d7c-4768-b82f-0555015cac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.175883 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c494d-2d7c-4768-b82f-0555015cac46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.176518 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c494d-2d7c-4768-b82f-0555015cac46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.178624 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c494d-2d7c-4768-b82f-0555015cac46 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.189736 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1421/2836] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:11 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:47:11.194950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481e09f-44c6-4aaa-bf18-65639b78b52e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.205175 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481e09f-44c6-4aaa-bf18-65639b78b52e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.210797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54952a34-aba5-413e-803b-dae7035e8487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.211448 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54952a34-aba5-413e-803b-dae7035e8487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.211724 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54952a34-aba5-413e-803b-dae7035e8487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.212000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54952a34-aba5-413e-803b-dae7035e8487 admin admin] PATH_INFO: `/v3/projects/3aeb63eec58343e486fd5d9443f049f5/users/d3c2302aad8d4eb0bf0038f70ba808ec/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.227322 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54952a34-aba5-413e-803b-dae7035e8487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aeb63eec58343e486fd5d9443f049f5, user_id=d3c2302aad8d4eb0bf0038f70ba808ec, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.229989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54952a34-aba5-413e-803b-dae7035e8487 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.243908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1416/2837] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:47:11 2022] PUT /identity/v3/projects/3aeb63eec58343e486fd5d9443f049f5/users/d3c2302aad8d4eb0bf0038f70ba808ec/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:11.254545 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8c83e-b20d-459e-af8b-23d6e7f157bc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.254830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8c83e-b20d-459e-af8b-23d6e7f157bc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.255103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8c83e-b20d-459e-af8b-23d6e7f157bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.286982 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-94c8c83e-b20d-459e-af8b-23d6e7f157bc None None] MFA Rules not processed for user `d3c2302aad8d4eb0bf0038f70ba808ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:11.288541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-94c8c83e-b20d-459e-af8b-23d6e7f157bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:11.360748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1422/2838] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:11.369257 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54952a34-aba5-413e-803b-dae7035e8487 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.379299 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:11.486200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54952a34-aba5-413e-803b-dae7035e8487 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.492171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84675f04-9fdf-47d0-b71b-a703d217f873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.492803 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84675f04-9fdf-47d0-b71b-a703d217f873 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.493080 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84675f04-9fdf-47d0-b71b-a703d217f873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.493354 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84675f04-9fdf-47d0-b71b-a703d217f873 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.494068 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84675f04-9fdf-47d0-b71b-a703d217f873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.495999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84675f04-9fdf-47d0-b71b-a703d217f873 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.527930 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1417/2839] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:47:11 2022] POST /identity/v3/projects => generated 382 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:11.532203 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8c83e-b20d-459e-af8b-23d6e7f157bc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.544911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8c83e-b20d-459e-af8b-23d6e7f157bc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.609435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1b55d-5393-4e2c-ab81-d313c06e7666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.610207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1b55d-5393-4e2c-ab81-d313c06e7666 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.610488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1b55d-5393-4e2c-ab81-d313c06e7666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.610758 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1b55d-5393-4e2c-ab81-d313c06e7666 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.611409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1b55d-5393-4e2c-ab81-d313c06e7666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.613362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1b55d-5393-4e2c-ab81-d313c06e7666 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.639920 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1423/2840] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:47:11 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:11.644473 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84675f04-9fdf-47d0-b71b-a703d217f873 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.656465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84675f04-9fdf-47d0-b71b-a703d217f873 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.662238 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc77b38-fcf8-487f-b592-bd32afe3c6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.662887 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc77b38-fcf8-487f-b592-bd32afe3c6ef admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.663174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc77b38-fcf8-487f-b592-bd32afe3c6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.663447 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc77b38-fcf8-487f-b592-bd32afe3c6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.664108 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc77b38-fcf8-487f-b592-bd32afe3c6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.665782 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc77b38-fcf8-487f-b592-bd32afe3c6ef admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.680987 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1418/2841] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:11 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:47:11.685673 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1b55d-5393-4e2c-ab81-d313c06e7666 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.692285 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:11.697459 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1b55d-5393-4e2c-ab81-d313c06e7666 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.703610 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f09684-a170-4f75-af42-be7728f95cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.704401 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f09684-a170-4f75-af42-be7728f95cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.704866 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f09684-a170-4f75-af42-be7728f95cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.705251 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f09684-a170-4f75-af42-be7728f95cb3 admin admin] PATH_INFO: `/v3/projects/c8e909bdcc5049eca1c233ca1e793657/users/e2a071a330814a4492053ed9f08bca86/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.722433 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f09684-a170-4f75-af42-be7728f95cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e909bdcc5049eca1c233ca1e793657, user_id=e2a071a330814a4492053ed9f08bca86, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.725279 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f09684-a170-4f75-af42-be7728f95cb3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.740787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1424/2842] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:47:11 2022] PUT /identity/v3/projects/c8e909bdcc5049eca1c233ca1e793657/users/e2a071a330814a4492053ed9f08bca86/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:11.745181 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc77b38-fcf8-487f-b592-bd32afe3c6ef admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.756274 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc77b38-fcf8-487f-b592-bd32afe3c6ef admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.818954 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33db1ae-dfaf-4588-b00b-e01278d162da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.819573 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33db1ae-dfaf-4588-b00b-e01278d162da admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.819860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33db1ae-dfaf-4588-b00b-e01278d162da admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.820130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33db1ae-dfaf-4588-b00b-e01278d162da admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.820808 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33db1ae-dfaf-4588-b00b-e01278d162da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.822472 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33db1ae-dfaf-4588-b00b-e01278d162da admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.833202 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1419/2843] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:11 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:47:11.838190 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f09684-a170-4f75-af42-be7728f95cb3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.849256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f09684-a170-4f75-af42-be7728f95cb3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.854047 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e5fd4-892b-4ace-9565-8a190e9d0c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.854708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e5fd4-892b-4ace-9565-8a190e9d0c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.854987 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e5fd4-892b-4ace-9565-8a190e9d0c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.855253 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e5fd4-892b-4ace-9565-8a190e9d0c31 admin admin] PATH_INFO: `/v3/projects/c8e909bdcc5049eca1c233ca1e793657/users/e2a071a330814a4492053ed9f08bca86/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.857200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e5fd4-892b-4ace-9565-8a190e9d0c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e909bdcc5049eca1c233ca1e793657, user_id=e2a071a330814a4492053ed9f08bca86, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.859858 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e5fd4-892b-4ace-9565-8a190e9d0c31 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.871544 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1425/2844] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:47:11 2022] PUT /identity/v3/projects/c8e909bdcc5049eca1c233ca1e793657/users/e2a071a330814a4492053ed9f08bca86/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:11.884627 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d51c9-c35a-47a9-a065-238884bc9369 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.884906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d51c9-c35a-47a9-a065-238884bc9369 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.885171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d51c9-c35a-47a9-a065-238884bc9369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.917867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b89d51c9-c35a-47a9-a065-238884bc9369 None None] MFA Rules not processed for user `e2a071a330814a4492053ed9f08bca86`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:11.919005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b89d51c9-c35a-47a9-a065-238884bc9369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:11.989115 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:11.995443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1420/2845] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:12.007437 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e5fd4-892b-4ace-9565-8a190e9d0c31 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:12.009677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c4e5fd4-892b-4ace-9565-8a190e9d0c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:12.071783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e5fd4-892b-4ace-9565-8a190e9d0c31 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:12.075557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8481a58-4074-4eb8-b8ce-b81d349cb8c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:12.076081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481a58-4074-4eb8-b8ce-b81d349cb8c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:12.076343 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481a58-4074-4eb8-b8ce-b81d349cb8c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:12.076611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481a58-4074-4eb8-b8ce-b81d349cb8c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:12.086159 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8481a58-4074-4eb8-b8ce-b81d349cb8c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:12.087598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8481a58-4074-4eb8-b8ce-b81d349cb8c8 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:12.097434 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1426/2846] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:12 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:12.176336 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d51c9-c35a-47a9-a065-238884bc9369 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:12.186793 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d51c9-c35a-47a9-a065-238884bc9369 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:12.190458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4000e-4c87-47d5-b4c0-5c4dcbb43cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:12.191089 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4000e-4c87-47d5-b4c0-5c4dcbb43cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:12.191478 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4000e-4c87-47d5-b4c0-5c4dcbb43cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:12.191723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4000e-4c87-47d5-b4c0-5c4dcbb43cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:12.252103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4000e-4c87-47d5-b4c0-5c4dcbb43cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:12.254171 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4000e-4c87-47d5-b4c0-5c4dcbb43cd3 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:12.264652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1421/2847] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:12 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:21.175554 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8481a58-4074-4eb8-b8ce-b81d349cb8c8 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:21.237477 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8481a58-4074-4eb8-b8ce-b81d349cb8c8 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:21.242962 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:21.243598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:21.243879 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:21.244150 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] PATH_INFO: `/v3/users/d3c2302aad8d4eb0bf0038f70ba808ec` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:21.245326 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c2302aad8d4eb0bf0038f70ba808ec)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:21.247345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:21.296865 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c2302aad8d4eb0bf0038f70ba808ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:21.309356 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c2302aad8d4eb0bf0038f70ba808ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:21.314494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:21.316533 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c2302aad8d4eb0bf0038f70ba808ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:21.325015 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c2302aad8d4eb0bf0038f70ba808ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:21.326122 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c2302aad8d4eb0bf0038f70ba808ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:21.332226 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1427/2848] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:47:21 2022] DELETE /identity/v3/users/d3c2302aad8d4eb0bf0038f70ba808ec => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:21.337592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7ce3d-77ad-4616-a160-38ec4da61e17 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:21.338221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7ce3d-77ad-4616-a160-38ec4da61e17 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:21.338221 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7ce3d-77ad-4616-a160-38ec4da61e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:21.357614 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a3c7ce3d-77ad-4616-a160-38ec4da61e17 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:21.358832 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a3c7ce3d-77ad-4616-a160-38ec4da61e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:21.423913 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1422/2849] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:21.433791 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:21.435209 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:21.493375 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3b3a7-8fd9-44e5-aa79-f0f4c3087407 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:21.497256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d0f02-cfe9-4f9e-a8a4-604e944a83fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:21.497906 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d0f02-cfe9-4f9e-a8a4-604e944a83fc service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:21.498189 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d0f02-cfe9-4f9e-a8a4-604e944a83fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:21.498462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d0f02-cfe9-4f9e-a8a4-604e944a83fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:21.509598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d0f02-cfe9-4f9e-a8a4-604e944a83fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:21.511041 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d0f02-cfe9-4f9e-a8a4-604e944a83fc service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:21.522988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1428/2850] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:22.061680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7ce3d-77ad-4616-a160-38ec4da61e17 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:22.063105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a3c7ce3d-77ad-4616-a160-38ec4da61e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:22.079560 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7ce3d-77ad-4616-a160-38ec4da61e17 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:22.086058 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:22.087206 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:22.087496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:22.087814 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] PATH_INFO: `/v3/projects/3aeb63eec58343e486fd5d9443f049f5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:22.089049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aeb63eec58343e486fd5d9443f049f5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:22.090970 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:22.144666 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aeb63eec58343e486fd5d9443f049f5', 'request_id': 'req-a5bcb403-30da-402a-82a8-6d8e72f58276'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.151991 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aeb63eec58343e486fd5d9443f049f5', 'request_id': 'req-a5bcb403-30da-402a-82a8-6d8e72f58276'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.158228 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aeb63eec58343e486fd5d9443f049f5', 'request_id': 'req-a5bcb403-30da-402a-82a8-6d8e72f58276'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.173164 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] The token cache is being invalidate because project 3aeb63eec58343e486fd5d9443f049f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:47:22.173443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.174182 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.175694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1423/2851] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:47:22 2022] DELETE /identity/v3/projects/3aeb63eec58343e486fd5d9443f049f5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:22.181264 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d0f02-cfe9-4f9e-a8a4-604e944a83fc service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:22.191956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d0f02-cfe9-4f9e-a8a4-604e944a83fc service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:22.270272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:22.271463 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:22.271740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:22.272013 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] PATH_INFO: `/v3/users/e2a071a330814a4492053ed9f08bca86` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:22.273354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a071a330814a4492053ed9f08bca86)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:22.275577 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:22.338067 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a071a330814a4492053ed9f08bca86'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.351945 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a071a330814a4492053ed9f08bca86'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.359200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a071a330814a4492053ed9f08bca86'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.368485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a071a330814a4492053ed9f08bca86'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.369121 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d71c7fc0-0f58-4747-afe6-c9342cf0bb67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a071a330814a4492053ed9f08bca86'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.375720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1429/2852] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:47:22 2022] DELETE /identity/v3/users/e2a071a330814a4492053ed9f08bca86 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:22.661862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:22.663470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:22.677583 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcb403-30da-402a-82a8-6d8e72f58276 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:22.699867 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:22.741084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:22.741748 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:22.742029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:22.742341 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] PATH_INFO: `/v3/projects/c8e909bdcc5049eca1c233ca1e793657` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:22.743459 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e909bdcc5049eca1c233ca1e793657)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:22.745359 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:22.793488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e909bdcc5049eca1c233ca1e793657', 'request_id': 'req-dba65202-410d-43ca-bcaa-11a20218baf9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.799557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e909bdcc5049eca1c233ca1e793657', 'request_id': 'req-dba65202-410d-43ca-bcaa-11a20218baf9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.805549 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e909bdcc5049eca1c233ca1e793657', 'request_id': 'req-dba65202-410d-43ca-bcaa-11a20218baf9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.818719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] The token cache is being invalidate because project c8e909bdcc5049eca1c233ca1e793657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:47:22.819034 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.819662 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-dba65202-410d-43ca-bcaa-11a20218baf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:22.821150 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1424/2853] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:47:22 2022] DELETE /identity/v3/projects/c8e909bdcc5049eca1c233ca1e793657 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:22.826376 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c0e44-4240-4010-9fe5-833ee561c78a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:22.826746 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c0e44-4240-4010-9fe5-833ee561c78a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:22.827087 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c0e44-4240-4010-9fe5-833ee561c78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:22.847429 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b25c0e44-4240-4010-9fe5-833ee561c78a None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:22.848543 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b25c0e44-4240-4010-9fe5-833ee561c78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:22.854903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:22.913498 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1430/2854] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:22.921766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a94d8-f3b4-4d96-8ade-8e4518197511 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:22.922055 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a94d8-f3b4-4d96-8ade-8e4518197511 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:22.922332 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a94d8-f3b4-4d96-8ade-8e4518197511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:22.941979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6d6a94d8-f3b4-4d96-8ade-8e4518197511 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:22.943229 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6d6a94d8-f3b4-4d96-8ade-8e4518197511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:22.951197 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1425/2855] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:47:22.955537 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c0e44-4240-4010-9fe5-833ee561c78a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:22.966141 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c0e44-4240-4010-9fe5-833ee561c78a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:22.970783 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5f550-8c25-4778-a67e-e3377bad8c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:22.971382 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5f550-8c25-4778-a67e-e3377bad8c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:22.971661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5f550-8c25-4778-a67e-e3377bad8c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:22.971936 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5f550-8c25-4778-a67e-e3377bad8c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:22.972594 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5f550-8c25-4778-a67e-e3377bad8c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:47:22.972892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5f550-8c25-4778-a67e-e3377bad8c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:22.975282 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5f550-8c25-4778-a67e-e3377bad8c82 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:22.986909 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1431/2856] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:47:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:22.991399 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a94d8-f3b4-4d96-8ade-8e4518197511 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.000566 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a94d8-f3b4-4d96-8ade-8e4518197511 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.005670 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b209c-6ad1-458d-9fa6-d5db8349d77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.006147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b209c-6ad1-458d-9fa6-d5db8349d77b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.006422 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b209c-6ad1-458d-9fa6-d5db8349d77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.006686 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b209c-6ad1-458d-9fa6-d5db8349d77b admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.007302 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b209c-6ad1-458d-9fa6-d5db8349d77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.009347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b209c-6ad1-458d-9fa6-d5db8349d77b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.037162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1426/2857] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:47:22 2022] POST /identity/v3/projects => generated 388 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:23.041471 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5f550-8c25-4778-a67e-e3377bad8c82 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.053366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5f550-8c25-4778-a67e-e3377bad8c82 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.111138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f97cea-7ced-4cc0-9828-111a0e5b74fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.111738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f97cea-7ced-4cc0-9828-111a0e5b74fd admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.112030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f97cea-7ced-4cc0-9828-111a0e5b74fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.112278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f97cea-7ced-4cc0-9828-111a0e5b74fd admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.112887 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f97cea-7ced-4cc0-9828-111a0e5b74fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.115661 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f97cea-7ced-4cc0-9828-111a0e5b74fd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.139177 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1432/2858] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:47:23 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:23.143445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b209c-6ad1-458d-9fa6-d5db8349d77b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.153942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b209c-6ad1-458d-9fa6-d5db8349d77b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.158528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692bfec-20b4-4fa2-883a-0983e5098838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.159118 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692bfec-20b4-4fa2-883a-0983e5098838 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.159393 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692bfec-20b4-4fa2-883a-0983e5098838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.159659 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692bfec-20b4-4fa2-883a-0983e5098838 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.160292 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692bfec-20b4-4fa2-883a-0983e5098838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.162496 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692bfec-20b4-4fa2-883a-0983e5098838 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.172335 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1427/2859] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:23 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:47:23.176296 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f97cea-7ced-4cc0-9828-111a0e5b74fd admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.187109 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f97cea-7ced-4cc0-9828-111a0e5b74fd admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.191706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bc2eb-37ff-487f-8115-d86ac3bced76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.192287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc2eb-37ff-487f-8115-d86ac3bced76 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.192562 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc2eb-37ff-487f-8115-d86ac3bced76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.192835 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc2eb-37ff-487f-8115-d86ac3bced76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.193550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bc2eb-37ff-487f-8115-d86ac3bced76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.195163 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bc2eb-37ff-487f-8115-d86ac3bced76 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.205893 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1433/2860] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:23 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:47:23.210010 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692bfec-20b4-4fa2-883a-0983e5098838 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.220254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692bfec-20b4-4fa2-883a-0983e5098838 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.226266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc378dcf-fb9c-4a7e-8383-de2cae03a0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.227060 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc378dcf-fb9c-4a7e-8383-de2cae03a0de admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.227371 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc378dcf-fb9c-4a7e-8383-de2cae03a0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.227673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc378dcf-fb9c-4a7e-8383-de2cae03a0de admin admin] PATH_INFO: `/v3/projects/5c6254c7d35c430ea1135fb9620916f7/users/20012a40cb6246f9aa1c007abd48dcb7/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.245219 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc378dcf-fb9c-4a7e-8383-de2cae03a0de admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6254c7d35c430ea1135fb9620916f7, user_id=20012a40cb6246f9aa1c007abd48dcb7, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.247908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc378dcf-fb9c-4a7e-8383-de2cae03a0de admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.262309 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1428/2861] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:47:23 2022] PUT /identity/v3/projects/5c6254c7d35c430ea1135fb9620916f7/users/20012a40cb6246f9aa1c007abd48dcb7/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:47:23.273795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b149b3-b268-4571-a8d2-3348dda7ac42 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.274081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b149b3-b268-4571-a8d2-3348dda7ac42 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.274358 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b149b3-b268-4571-a8d2-3348dda7ac42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.303485 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f7b149b3-b268-4571-a8d2-3348dda7ac42 None None] MFA Rules not processed for user `20012a40cb6246f9aa1c007abd48dcb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:23.304697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f7b149b3-b268-4571-a8d2-3348dda7ac42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:23.369056 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1434/2862] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:23.373475 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc378dcf-fb9c-4a7e-8383-de2cae03a0de admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.422103 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:23.437836 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc378dcf-fb9c-4a7e-8383-de2cae03a0de admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.442630 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da506d-ec88-4515-b417-cee7de69092e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.443249 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da506d-ec88-4515-b417-cee7de69092e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.443533 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da506d-ec88-4515-b417-cee7de69092e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.443806 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da506d-ec88-4515-b417-cee7de69092e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.444457 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da506d-ec88-4515-b417-cee7de69092e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.446381 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da506d-ec88-4515-b417-cee7de69092e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.472024 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1429/2863] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:47:23 2022] POST /identity/v3/projects => generated 388 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:23.476732 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b149b3-b268-4571-a8d2-3348dda7ac42 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.487591 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b149b3-b268-4571-a8d2-3348dda7ac42 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.491708 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:23.549813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600337d-56fd-48c4-b0da-00539c56ad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.550418 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600337d-56fd-48c4-b0da-00539c56ad74 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.550697 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600337d-56fd-48c4-b0da-00539c56ad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.550966 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600337d-56fd-48c4-b0da-00539c56ad74 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.551601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600337d-56fd-48c4-b0da-00539c56ad74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.553659 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600337d-56fd-48c4-b0da-00539c56ad74 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.575239 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1435/2864] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:47:23 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:23.579315 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da506d-ec88-4515-b417-cee7de69092e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.590950 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da506d-ec88-4515-b417-cee7de69092e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.595660 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d695b-cc10-4eda-a984-a0a5763c5dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.596258 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d695b-cc10-4eda-a984-a0a5763c5dda admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.596537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d695b-cc10-4eda-a984-a0a5763c5dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.596811 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d695b-cc10-4eda-a984-a0a5763c5dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.597440 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d695b-cc10-4eda-a984-a0a5763c5dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.599842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d695b-cc10-4eda-a984-a0a5763c5dda admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.610357 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1430/2865] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:23 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:47:23.614579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600337d-56fd-48c4-b0da-00539c56ad74 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.625458 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600337d-56fd-48c4-b0da-00539c56ad74 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.630006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaca47-fc3b-45fd-a00a-4280281f08ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.630653 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaca47-fc3b-45fd-a00a-4280281f08ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.630923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaca47-fc3b-45fd-a00a-4280281f08ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.631223 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaca47-fc3b-45fd-a00a-4280281f08ba admin admin] PATH_INFO: `/v3/projects/55c3606c2d644a69a4e9a8f7eaa33913/users/984de9403cc84f1284ebbf8a637e422f/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.646795 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaca47-fc3b-45fd-a00a-4280281f08ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c3606c2d644a69a4e9a8f7eaa33913, user_id=984de9403cc84f1284ebbf8a637e422f, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.649432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaca47-fc3b-45fd-a00a-4280281f08ba admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.666703 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1436/2866] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:47:23 2022] PUT /identity/v3/projects/55c3606c2d644a69a4e9a8f7eaa33913/users/984de9403cc84f1284ebbf8a637e422f/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:23.670865 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d695b-cc10-4eda-a984-a0a5763c5dda admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.680895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d695b-cc10-4eda-a984-a0a5763c5dda admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.738917 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761d149-50ec-401c-8929-9312d2729672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.739797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761d149-50ec-401c-8929-9312d2729672 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.740076 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761d149-50ec-401c-8929-9312d2729672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.740356 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761d149-50ec-401c-8929-9312d2729672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.740993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761d149-50ec-401c-8929-9312d2729672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.742616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761d149-50ec-401c-8929-9312d2729672 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.752545 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1431/2867] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:23 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:47:23.757027 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaca47-fc3b-45fd-a00a-4280281f08ba admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.767479 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaca47-fc3b-45fd-a00a-4280281f08ba admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.772637 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2033ea2-0388-49a1-b61b-5d52d85ca8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.773288 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2033ea2-0388-49a1-b61b-5d52d85ca8d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.773589 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2033ea2-0388-49a1-b61b-5d52d85ca8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.773873 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2033ea2-0388-49a1-b61b-5d52d85ca8d7 admin admin] PATH_INFO: `/v3/projects/55c3606c2d644a69a4e9a8f7eaa33913/users/984de9403cc84f1284ebbf8a637e422f/roles/8857e6335b964c8ba99f8b0499179c75` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.775995 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2033ea2-0388-49a1-b61b-5d52d85ca8d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c3606c2d644a69a4e9a8f7eaa33913, user_id=984de9403cc84f1284ebbf8a637e422f, role_id=8857e6335b964c8ba99f8b0499179c75)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.778689 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2033ea2-0388-49a1-b61b-5d52d85ca8d7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:23.792743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1437/2868] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:47:23 2022] PUT /identity/v3/projects/55c3606c2d644a69a4e9a8f7eaa33913/users/984de9403cc84f1284ebbf8a637e422f/roles/8857e6335b964c8ba99f8b0499179c75 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:23.804398 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bf37e-c9ee-4cd7-9c06-91df44a8b2f8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.804732 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bf37e-c9ee-4cd7-9c06-91df44a8b2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.805005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bf37e-c9ee-4cd7-9c06-91df44a8b2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.832528 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2f3bf37e-c9ee-4cd7-9c06-91df44a8b2f8 None None] MFA Rules not processed for user `984de9403cc84f1284ebbf8a637e422f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:23.834564 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2f3bf37e-c9ee-4cd7-9c06-91df44a8b2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:23.905694 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1432/2869] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:23.917359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2033ea2-0388-49a1-b61b-5d52d85ca8d7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:23.919196 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e2033ea2-0388-49a1-b61b-5d52d85ca8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:23.976541 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2033ea2-0388-49a1-b61b-5d52d85ca8d7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:23.980200 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b8b5b-6e70-4200-923e-57a0e13a3540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:23.980794 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b8b5b-6e70-4200-923e-57a0e13a3540 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:23.981069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b8b5b-6e70-4200-923e-57a0e13a3540 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:23.981332 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b8b5b-6e70-4200-923e-57a0e13a3540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:23.990482 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b8b5b-6e70-4200-923e-57a0e13a3540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:23.991913 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b8b5b-6e70-4200-923e-57a0e13a3540 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:24.001409 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1438/2870] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:23 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:24.031105 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bf37e-c9ee-4cd7-9c06-91df44a8b2f8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:24.076602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:24.095854 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bf37e-c9ee-4cd7-9c06-91df44a8b2f8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:24.101191 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:24.101860 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:24.102140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:24.102411 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] PATH_INFO: `/v3/users/20012a40cb6246f9aa1c007abd48dcb7` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:24.103552 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20012a40cb6246f9aa1c007abd48dcb7)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:24.105466 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:24.153470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20012a40cb6246f9aa1c007abd48dcb7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.160873 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20012a40cb6246f9aa1c007abd48dcb7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.161463 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20012a40cb6246f9aa1c007abd48dcb7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.173426 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20012a40cb6246f9aa1c007abd48dcb7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.180177 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20012a40cb6246f9aa1c007abd48dcb7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.186420 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1433/2871] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:47:24 2022] DELETE /identity/v3/users/20012a40cb6246f9aa1c007abd48dcb7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:24.190899 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae699db-b052-4fe8-9480-7b0b9adcdf5b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:24.191366 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae699db-b052-4fe8-9480-7b0b9adcdf5b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:24.191466 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae699db-b052-4fe8-9480-7b0b9adcdf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:24.211039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fae699db-b052-4fe8-9480-7b0b9adcdf5b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:24.212229 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fae699db-b052-4fe8-9480-7b0b9adcdf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:24.269652 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1439/2872] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:24.279707 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:24.281187 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:24.346812 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499860e-93f7-4929-a7bf-10717a97a664 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:24.351162 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28625c8d-f36e-44c1-8a95-989bc39cda9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:24.351770 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28625c8d-f36e-44c1-8a95-989bc39cda9d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:24.352044 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28625c8d-f36e-44c1-8a95-989bc39cda9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:24.352311 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28625c8d-f36e-44c1-8a95-989bc39cda9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:24.361842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28625c8d-f36e-44c1-8a95-989bc39cda9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:24.363269 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28625c8d-f36e-44c1-8a95-989bc39cda9d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:24.374571 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1434/2873] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:24.679800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae699db-b052-4fe8-9480-7b0b9adcdf5b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:24.681019 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fae699db-b052-4fe8-9480-7b0b9adcdf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:24.689053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:24.696002 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae699db-b052-4fe8-9480-7b0b9adcdf5b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:24.701694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:24.702354 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:24.702635 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:24.702908 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] PATH_INFO: `/v3/projects/5c6254c7d35c430ea1135fb9620916f7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:24.704261 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6254c7d35c430ea1135fb9620916f7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:24.706220 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:24.755695 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6254c7d35c430ea1135fb9620916f7', 'request_id': 'req-e9c977f5-8f90-4b4a-b721-5b35ae422044'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.763557 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6254c7d35c430ea1135fb9620916f7', 'request_id': 'req-e9c977f5-8f90-4b4a-b721-5b35ae422044'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.769513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6254c7d35c430ea1135fb9620916f7', 'request_id': 'req-e9c977f5-8f90-4b4a-b721-5b35ae422044'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.781611 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] The token cache is being invalidate because project 5c6254c7d35c430ea1135fb9620916f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:47:24.781898 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.782558 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.784241 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1440/2874] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:47:24 2022] DELETE /identity/v3/projects/5c6254c7d35c430ea1135fb9620916f7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:24.789145 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28625c8d-f36e-44c1-8a95-989bc39cda9d service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:24.799141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28625c8d-f36e-44c1-8a95-989bc39cda9d service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:24.857443 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:24.858107 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:24.858380 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:24.859271 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] PATH_INFO: `/v3/users/984de9403cc84f1284ebbf8a637e422f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:24.859794 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=984de9403cc84f1284ebbf8a637e422f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:24.861730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:24.868764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:24.913696 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '984de9403cc84f1284ebbf8a637e422f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.929604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '984de9403cc84f1284ebbf8a637e422f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.929604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '984de9403cc84f1284ebbf8a637e422f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.938569 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '984de9403cc84f1284ebbf8a637e422f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.945642 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92e2c64c-8e2e-4b15-a790-681163c9a6f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '984de9403cc84f1284ebbf8a637e422f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:24.951909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1435/2875] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:47:24 2022] DELETE /identity/v3/users/984de9403cc84f1284ebbf8a637e422f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:25.256059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:25.257552 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:25.271688 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c977f5-8f90-4b4a-b721-5b35ae422044 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:25.342767 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:25.343323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.343604 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.343889 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] PATH_INFO: `/v3/projects/55c3606c2d644a69a4e9a8f7eaa33913` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.345034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c3606c2d644a69a4e9a8f7eaa33913)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:25.347001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:25.403943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c3606c2d644a69a4e9a8f7eaa33913', 'request_id': 'req-302fe89c-ebab-450c-9dfe-59067cd46efb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:25.412476 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c3606c2d644a69a4e9a8f7eaa33913', 'request_id': 'req-302fe89c-ebab-450c-9dfe-59067cd46efb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:25.418578 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c3606c2d644a69a4e9a8f7eaa33913', 'request_id': 'req-302fe89c-ebab-450c-9dfe-59067cd46efb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:25.430506 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] The token cache is being invalidate because project 55c3606c2d644a69a4e9a8f7eaa33913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:47:25.431186 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:25.431778 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-302fe89c-ebab-450c-9dfe-59067cd46efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:25.433265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1441/2876] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:47:25 2022] DELETE /identity/v3/projects/55c3606c2d644a69a4e9a8f7eaa33913 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:47:25.438138 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93504fe9-3c5f-43f0-bf1f-e9fdcf483759 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.438561 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93504fe9-3c5f-43f0-bf1f-e9fdcf483759 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.438885 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93504fe9-3c5f-43f0-bf1f-e9fdcf483759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.463700 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-93504fe9-3c5f-43f0-bf1f-e9fdcf483759 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:25.464923 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-93504fe9-3c5f-43f0-bf1f-e9fdcf483759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:25.534053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1436/2877] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:25.543424 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee2b4b-7f2c-4a39-b3c1-f4e790cdb9a6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.543955 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee2b4b-7f2c-4a39-b3c1-f4e790cdb9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.544439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee2b4b-7f2c-4a39-b3c1-f4e790cdb9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.568570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-57ee2b4b-7f2c-4a39-b3c1-f4e790cdb9a6 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:25.570781 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-57ee2b4b-7f2c-4a39-b3c1-f4e790cdb9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:25.585069 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1442/2878] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:25.590241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93504fe9-3c5f-43f0-bf1f-e9fdcf483759 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:25.600656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93504fe9-3c5f-43f0-bf1f-e9fdcf483759 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:25.605185 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b173296-0d16-441b-b450-477b9308f1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:25.605841 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b173296-0d16-441b-b450-477b9308f1fb admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.606119 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b173296-0d16-441b-b450-477b9308f1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.606389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b173296-0d16-441b-b450-477b9308f1fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.607048 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b173296-0d16-441b-b450-477b9308f1fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:47:25.607350 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b173296-0d16-441b-b450-477b9308f1fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:25.608993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b173296-0d16-441b-b450-477b9308f1fb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:25.621387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1437/2879] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:47:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:25.628707 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee2b4b-7f2c-4a39-b3c1-f4e790cdb9a6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:25.639435 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee2b4b-7f2c-4a39-b3c1-f4e790cdb9a6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:25.644412 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d0aed-2602-40ee-958e-b738315a3b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:25.645031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d0aed-2602-40ee-958e-b738315a3b45 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.645313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d0aed-2602-40ee-958e-b738315a3b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.646256 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d0aed-2602-40ee-958e-b738315a3b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.647065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d0aed-2602-40ee-958e-b738315a3b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:25.648854 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d0aed-2602-40ee-958e-b738315a3b45 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:25.654811 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:25.681606 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1443/2880] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:47:25 2022] POST /identity/v3/projects => generated 378 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:25.686067 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b173296-0d16-441b-b450-477b9308f1fb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:25.697521 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b173296-0d16-441b-b450-477b9308f1fb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:25.758389 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59279d-3bdd-485e-a640-ee31c6da7f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:25.759327 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59279d-3bdd-485e-a640-ee31c6da7f3d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.759602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59279d-3bdd-485e-a640-ee31c6da7f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.759872 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59279d-3bdd-485e-a640-ee31c6da7f3d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.760503 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59279d-3bdd-485e-a640-ee31c6da7f3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:25.762464 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59279d-3bdd-485e-a640-ee31c6da7f3d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:25.785347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1438/2881] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:47:25 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:25.790100 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d0aed-2602-40ee-958e-b738315a3b45 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:25.802147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d0aed-2602-40ee-958e-b738315a3b45 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:25.807903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbdc78-1d21-45ff-a62e-7711de24248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:25.808516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbdc78-1d21-45ff-a62e-7711de24248f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.808793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbdc78-1d21-45ff-a62e-7711de24248f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.809065 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbdc78-1d21-45ff-a62e-7711de24248f admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.809745 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbdc78-1d21-45ff-a62e-7711de24248f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:25.811363 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbdc78-1d21-45ff-a62e-7711de24248f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:25.823495 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1444/2882] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:25 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:47:25.827695 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59279d-3bdd-485e-a640-ee31c6da7f3d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:25.841953 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:25.848278 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59279d-3bdd-485e-a640-ee31c6da7f3d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:25.854385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f9302-48c6-4f3c-acc4-aa205a03c4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:25.854926 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f9302-48c6-4f3c-acc4-aa205a03c4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.855213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f9302-48c6-4f3c-acc4-aa205a03c4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.855491 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f9302-48c6-4f3c-acc4-aa205a03c4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.856140 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f9302-48c6-4f3c-acc4-aa205a03c4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:25.857792 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f9302-48c6-4f3c-acc4-aa205a03c4c8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:25.869862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1439/2883] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:25 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:47:25.874598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbdc78-1d21-45ff-a62e-7711de24248f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:25.886143 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbdc78-1d21-45ff-a62e-7711de24248f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:25.890940 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf01210-b53a-4181-95b5-9a9f1c17c37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:25.891613 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf01210-b53a-4181-95b5-9a9f1c17c37e admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.891885 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf01210-b53a-4181-95b5-9a9f1c17c37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.892151 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf01210-b53a-4181-95b5-9a9f1c17c37e admin admin] PATH_INFO: `/v3/projects/315f50ec729a420d836fa4c5d69897f0/users/05df702730ae4c97995e0eb02153a803/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.907888 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf01210-b53a-4181-95b5-9a9f1c17c37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=315f50ec729a420d836fa4c5d69897f0, user_id=05df702730ae4c97995e0eb02153a803, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:25.910603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf01210-b53a-4181-95b5-9a9f1c17c37e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:25.925740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1445/2884] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:47:25 2022] PUT /identity/v3/projects/315f50ec729a420d836fa4c5d69897f0/users/05df702730ae4c97995e0eb02153a803/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:25.938537 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7811759-8f90-4cd0-9ec4-fd8a6e7622a1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:25.938821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7811759-8f90-4cd0-9ec4-fd8a6e7622a1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:25.939092 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7811759-8f90-4cd0-9ec4-fd8a6e7622a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:25.969813 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f7811759-8f90-4cd0-9ec4-fd8a6e7622a1 None None] MFA Rules not processed for user `05df702730ae4c97995e0eb02153a803`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:25.971026 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f7811759-8f90-4cd0-9ec4-fd8a6e7622a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:26.047602 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1440/2885] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:26.060044 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf01210-b53a-4181-95b5-9a9f1c17c37e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:26.061833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2cf01210-b53a-4181-95b5-9a9f1c17c37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:26.128199 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf01210-b53a-4181-95b5-9a9f1c17c37e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:26.133222 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf5871-56c1-43dd-8827-bdfb0c736b9c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0891d81e6e24ad99a9b532059c8c2af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:26.134008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf5871-56c1-43dd-8827-bdfb0c736b9c service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:26.134194 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf5871-56c1-43dd-8827-bdfb0c736b9c service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:26.134428 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf5871-56c1-43dd-8827-bdfb0c736b9c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:26.146923 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf5871-56c1-43dd-8827-bdfb0c736b9c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:26.148374 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf5871-56c1-43dd-8827-bdfb0c736b9c service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:26.157821 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1446/2886] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:36.044646 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=36 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:36.153512 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:44.078071 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7811759-8f90-4cd0-9ec4-fd8a6e7622a1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:44.148388 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7811759-8f90-4cd0-9ec4-fd8a6e7622a1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:44.153052 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:44.153737 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:44.154016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:44.154290 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] PATH_INFO: `/v3/users/05df702730ae4c97995e0eb02153a803` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:44.155460 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=05df702730ae4c97995e0eb02153a803)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:44.157385 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:44.209809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05df702730ae4c97995e0eb02153a803'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.217669 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05df702730ae4c97995e0eb02153a803'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.218321 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05df702730ae4c97995e0eb02153a803'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.232539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05df702730ae4c97995e0eb02153a803'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.240706 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05df702730ae4c97995e0eb02153a803'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.247130 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1441/2887] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:47:44 2022] DELETE /identity/v3/users/05df702730ae4c97995e0eb02153a803 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:44.251517 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ed7e-802e-47cd-910e-a5cfabea0c1b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:44.252127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ed7e-802e-47cd-910e-a5cfabea0c1b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:44.252127 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ed7e-802e-47cd-910e-a5cfabea0c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:44.272140 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7621ed7e-802e-47cd-910e-a5cfabea0c1b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:44.273385 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7621ed7e-802e-47cd-910e-a5cfabea0c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:44.335787 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1447/2888] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:44.345755 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:44.347250 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:44.411141 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fed4b7-54f4-4f3c-a8dc-31ec49e828bf admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:44.414992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522c5d0-7203-42b9-99ce-5bb0732b2c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:44.415604 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522c5d0-7203-42b9-99ce-5bb0732b2c8a service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:44.415881 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522c5d0-7203-42b9-99ce-5bb0732b2c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:44.416147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522c5d0-7203-42b9-99ce-5bb0732b2c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:44.426042 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522c5d0-7203-42b9-99ce-5bb0732b2c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:44.428468 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522c5d0-7203-42b9-99ce-5bb0732b2c8a service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:44.438414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1442/2889] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:44.739789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621ed7e-802e-47cd-910e-a5cfabea0c1b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:44.741323 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7621ed7e-802e-47cd-910e-a5cfabea0c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:44.754479 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621ed7e-802e-47cd-910e-a5cfabea0c1b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:44.759943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:44.760599 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:44.760882 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:44.761156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] PATH_INFO: `/v3/projects/315f50ec729a420d836fa4c5d69897f0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:44.762381 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=315f50ec729a420d836fa4c5d69897f0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:44.764355 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:44.816457 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315f50ec729a420d836fa4c5d69897f0', 'request_id': 'req-404ccc0d-3a84-4e9c-971d-3821b61736f6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.824601 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315f50ec729a420d836fa4c5d69897f0', 'request_id': 'req-404ccc0d-3a84-4e9c-971d-3821b61736f6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.830481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315f50ec729a420d836fa4c5d69897f0', 'request_id': 'req-404ccc0d-3a84-4e9c-971d-3821b61736f6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.842629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] The token cache is being invalidate because project 315f50ec729a420d836fa4c5d69897f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:47:44.842986 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.843574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-404ccc0d-3a84-4e9c-971d-3821b61736f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:47:44.845053 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1448/2890] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:47:44 2022] DELETE /identity/v3/projects/315f50ec729a420d836fa4c5d69897f0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:44.851338 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90663f8e-237c-471b-a5e4-838da61efbfe None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:44.851621 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90663f8e-237c-471b-a5e4-838da61efbfe None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:44.851888 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90663f8e-237c-471b-a5e4-838da61efbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:44.873704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-90663f8e-237c-471b-a5e4-838da61efbfe None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:44.874905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-90663f8e-237c-471b-a5e4-838da61efbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:44.893730 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:44.941009 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1443/2891] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:44.949373 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fd0e9-bd16-4efd-9b5d-f960a9ad837b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:44.949681 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fd0e9-bd16-4efd-9b5d-f960a9ad837b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:44.949949 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fd0e9-bd16-4efd-9b5d-f960a9ad837b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:44.970294 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-457fd0e9-bd16-4efd-9b5d-f960a9ad837b None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:44.971521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-457fd0e9-bd16-4efd-9b5d-f960a9ad837b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:44.979903 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1449/2892] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:44.984279 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90663f8e-237c-471b-a5e4-838da61efbfe None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:44.994940 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90663f8e-237c-471b-a5e4-838da61efbfe None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:44.999718 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37788a0-0e6b-4677-b2af-40af03e5309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:45.000316 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37788a0-0e6b-4677-b2af-40af03e5309a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:45.000594 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37788a0-0e6b-4677-b2af-40af03e5309a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:45.000862 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37788a0-0e6b-4677-b2af-40af03e5309a admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:45.001557 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37788a0-0e6b-4677-b2af-40af03e5309a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:47:45.001884 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37788a0-0e6b-4677-b2af-40af03e5309a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:45.003455 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37788a0-0e6b-4677-b2af-40af03e5309a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:45.015550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1444/2893] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:47:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:45.020831 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fd0e9-bd16-4efd-9b5d-f960a9ad837b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:45.033064 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fd0e9-bd16-4efd-9b5d-f960a9ad837b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:45.038123 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deaefd-bc1b-43c9-baaa-e78c975c5403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:45.038725 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deaefd-bc1b-43c9-baaa-e78c975c5403 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:45.039001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deaefd-bc1b-43c9-baaa-e78c975c5403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:45.039268 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deaefd-bc1b-43c9-baaa-e78c975c5403 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:45.039891 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86deaefd-bc1b-43c9-baaa-e78c975c5403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:45.041801 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86deaefd-bc1b-43c9-baaa-e78c975c5403 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:45.070521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1450/2894] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:47:45 2022] POST /identity/v3/projects => generated 372 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:45.074883 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37788a0-0e6b-4677-b2af-40af03e5309a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:45.085289 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37788a0-0e6b-4677-b2af-40af03e5309a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:45.140394 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5daf98e-2448-4308-8dbc-0adc00ca4aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:45.140993 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5daf98e-2448-4308-8dbc-0adc00ca4aae admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:45.141267 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5daf98e-2448-4308-8dbc-0adc00ca4aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:45.141532 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5daf98e-2448-4308-8dbc-0adc00ca4aae admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:45.142180 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5daf98e-2448-4308-8dbc-0adc00ca4aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:45.144066 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5daf98e-2448-4308-8dbc-0adc00ca4aae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:45.166117 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1445/2895] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:47:45 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:45.170937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deaefd-bc1b-43c9-baaa-e78c975c5403 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:45.182035 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deaefd-bc1b-43c9-baaa-e78c975c5403 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:45.186627 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707385e-b18b-4ded-b9e2-5e2bf8e7d8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:45.187215 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707385e-b18b-4ded-b9e2-5e2bf8e7d8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:45.187494 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707385e-b18b-4ded-b9e2-5e2bf8e7d8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:45.187763 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707385e-b18b-4ded-b9e2-5e2bf8e7d8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:45.188462 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707385e-b18b-4ded-b9e2-5e2bf8e7d8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:45.190103 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707385e-b18b-4ded-b9e2-5e2bf8e7d8c7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:45.200603 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1451/2896] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:45 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:47:45.204786 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5daf98e-2448-4308-8dbc-0adc00ca4aae admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:45.214818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5daf98e-2448-4308-8dbc-0adc00ca4aae admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:45.219456 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205c819-c7b7-4c33-a6cb-17f4ee9f005c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:45.220041 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205c819-c7b7-4c33-a6cb-17f4ee9f005c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:45.220317 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205c819-c7b7-4c33-a6cb-17f4ee9f005c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:45.220587 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205c819-c7b7-4c33-a6cb-17f4ee9f005c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:45.221213 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205c819-c7b7-4c33-a6cb-17f4ee9f005c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:45.222824 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205c819-c7b7-4c33-a6cb-17f4ee9f005c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:45.233968 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1446/2897] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:47:45 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:47:45.238147 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707385e-b18b-4ded-b9e2-5e2bf8e7d8c7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:45.249030 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707385e-b18b-4ded-b9e2-5e2bf8e7d8c7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:45.253789 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c70b4b-e3d6-4573-8b5f-8a69ec4ca2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:45.254511 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c70b4b-e3d6-4573-8b5f-8a69ec4ca2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:45.254727 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c70b4b-e3d6-4573-8b5f-8a69ec4ca2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:45.255006 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c70b4b-e3d6-4573-8b5f-8a69ec4ca2f2 admin admin] PATH_INFO: `/v3/projects/7ca3e30a13b24738bb25c78a264ec3e7/users/04d79e9bd3944cc185e2b4d6ff703216/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:45.269484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c70b4b-e3d6-4573-8b5f-8a69ec4ca2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca3e30a13b24738bb25c78a264ec3e7, user_id=04d79e9bd3944cc185e2b4d6ff703216, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:45.272738 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c70b4b-e3d6-4573-8b5f-8a69ec4ca2f2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:45.286344 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1452/2898] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:47:45 2022] PUT /identity/v3/projects/7ca3e30a13b24738bb25c78a264ec3e7/users/04d79e9bd3944cc185e2b4d6ff703216/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:45.296570 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ca8a9-1017-45ce-9f55-514393d4b7be None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:45.296880 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ca8a9-1017-45ce-9f55-514393d4b7be None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:45.297152 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ca8a9-1017-45ce-9f55-514393d4b7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:45.325652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4d1ca8a9-1017-45ce-9f55-514393d4b7be None None] MFA Rules not processed for user `04d79e9bd3944cc185e2b4d6ff703216`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:45.326856 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4d1ca8a9-1017-45ce-9f55-514393d4b7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:45.390895 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1447/2899] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:47:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:47:45.402740 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c70b4b-e3d6-4573-8b5f-8a69ec4ca2f2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:45.404403 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c6c70b4b-e3d6-4573-8b5f-8a69ec4ca2f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:45.429003 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:45.487049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c70b4b-e3d6-4573-8b5f-8a69ec4ca2f2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:45.491579 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948d888-f2db-40ad-b0f8-26663000f839 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0891d81e6e24ad99a9b532059c8c2af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:45.492207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d888-f2db-40ad-b0f8-26663000f839 service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:45.492483 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d888-f2db-40ad-b0f8-26663000f839 service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:45.492749 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d888-f2db-40ad-b0f8-26663000f839 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:45.501693 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948d888-f2db-40ad-b0f8-26663000f839 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:45.503117 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948d888-f2db-40ad-b0f8-26663000f839 service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:45.512359 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1453/2900] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:51.514134 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ca8a9-1017-45ce-9f55-514393d4b7be None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:51.515740 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4d1ca8a9-1017-45ce-9f55-514393d4b7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:51.539992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:51.580215 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ca8a9-1017-45ce-9f55-514393d4b7be None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:51.584595 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b88ea-c003-49f7-88bf-12bcc49de1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:51.585218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b88ea-c003-49f7-88bf-12bcc49de1af service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:51.585500 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b88ea-c003-49f7-88bf-12bcc49de1af service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:51.585810 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b88ea-c003-49f7-88bf-12bcc49de1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:51.587578 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-546b88ea-c003-49f7-88bf-12bcc49de1af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:51.598592 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b88ea-c003-49f7-88bf-12bcc49de1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:51.600133 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b88ea-c003-49f7-88bf-12bcc49de1af service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:51.609207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1448/2901] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:47:51 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:47:55.508049 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=26 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:01.605125 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=18 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:12.423880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948d888-f2db-40ad-b0f8-26663000f839 service glance] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:12.491580 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948d888-f2db-40ad-b0f8-26663000f839 service glance] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:12.496130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:12.496758 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:12.497039 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:12.497345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] PATH_INFO: `/v3/users/04d79e9bd3944cc185e2b4d6ff703216` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:12.498514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d79e9bd3944cc185e2b4d6ff703216)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:12.500430 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:12.552272 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d79e9bd3944cc185e2b4d6ff703216'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:12.564489 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d79e9bd3944cc185e2b4d6ff703216'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:12.577504 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d79e9bd3944cc185e2b4d6ff703216'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:12.586700 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d79e9bd3944cc185e2b4d6ff703216'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:12.587536 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d79e9bd3944cc185e2b4d6ff703216'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:12.595625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1454/2902] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:48:12 2022] DELETE /identity/v3/users/04d79e9bd3944cc185e2b4d6ff703216 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:48:12.601567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551eca9c-1e95-4293-9c54-f42c27a3a5d4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:12.602113 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551eca9c-1e95-4293-9c54-f42c27a3a5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:12.602244 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551eca9c-1e95-4293-9c54-f42c27a3a5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:12.622029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-551eca9c-1e95-4293-9c54-f42c27a3a5d4 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:12.623554 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-551eca9c-1e95-4293-9c54-f42c27a3a5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:12.684684 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1449/2903] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:48:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:48:12.695142 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:12.696585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:12.755988 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb88dca-927d-468e-a2a1-97c6feaaaa91 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:12.761391 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531dbce-6b10-4f2e-b2c7-bd184f3c76c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:12.762106 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531dbce-6b10-4f2e-b2c7-bd184f3c76c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:12.762308 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531dbce-6b10-4f2e-b2c7-bd184f3c76c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:12.762574 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531dbce-6b10-4f2e-b2c7-bd184f3c76c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:12.772529 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531dbce-6b10-4f2e-b2c7-bd184f3c76c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:12.774001 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531dbce-6b10-4f2e-b2c7-bd184f3c76c1 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:12.783267 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1455/2904] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:48:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:48:13.066656 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551eca9c-1e95-4293-9c54-f42c27a3a5d4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:13.068242 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-551eca9c-1e95-4293-9c54-f42c27a3a5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:13.080254 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551eca9c-1e95-4293-9c54-f42c27a3a5d4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:13.086418 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:13.087310 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.087657 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.088014 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] PATH_INFO: `/v3/projects/7ca3e30a13b24738bb25c78a264ec3e7` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.089673 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca3e30a13b24738bb25c78a264ec3e7)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:13.092909 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:13.141511 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca3e30a13b24738bb25c78a264ec3e7', 'request_id': 'req-9d51232a-df8e-487b-944e-a5641d9dffd4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:13.148681 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca3e30a13b24738bb25c78a264ec3e7', 'request_id': 'req-9d51232a-df8e-487b-944e-a5641d9dffd4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:13.154452 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca3e30a13b24738bb25c78a264ec3e7', 'request_id': 'req-9d51232a-df8e-487b-944e-a5641d9dffd4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:13.168687 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] The token cache is being invalidate because project 7ca3e30a13b24738bb25c78a264ec3e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:48:13.169129 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:13.169893 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9d51232a-df8e-487b-944e-a5641d9dffd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:13.171369 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1450/2905] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:48:13 2022] DELETE /identity/v3/projects/7ca3e30a13b24738bb25c78a264ec3e7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:48:13.176278 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579794e-ec11-43b7-b292-8946f1d6bad8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.176556 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579794e-ec11-43b7-b292-8946f1d6bad8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.176850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579794e-ec11-43b7-b292-8946f1d6bad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.196819 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b579794e-ec11-43b7-b292-8946f1d6bad8 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:13.198113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b579794e-ec11-43b7-b292-8946f1d6bad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:13.215383 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:13.260144 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1456/2906] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:48:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:48:13.267832 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d6e6c-8b39-441b-80b3-5e61cb4be767 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.268116 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d6e6c-8b39-441b-80b3-5e61cb4be767 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.268387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d6e6c-8b39-441b-80b3-5e61cb4be767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.289432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-301d6e6c-8b39-441b-80b3-5e61cb4be767 None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:13.290652 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-301d6e6c-8b39-441b-80b3-5e61cb4be767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:13.298783 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1451/2907] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:48:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:48:13.303405 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579794e-ec11-43b7-b292-8946f1d6bad8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:13.312870 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579794e-ec11-43b7-b292-8946f1d6bad8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:13.317414 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2afd4-be23-4cce-8325-907319403857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:13.318311 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2afd4-be23-4cce-8325-907319403857 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.318585 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2afd4-be23-4cce-8325-907319403857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.318850 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2afd4-be23-4cce-8325-907319403857 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.319484 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2afd4-be23-4cce-8325-907319403857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:13.319780 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2afd4-be23-4cce-8325-907319403857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:13.321345 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2afd4-be23-4cce-8325-907319403857 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:13.332530 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1457/2908] 10.222.0.100 () {58 vars in 1158 bytes} [Thu Sep 1 17:48:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:13.337414 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d6e6c-8b39-441b-80b3-5e61cb4be767 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:13.347053 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d6e6c-8b39-441b-80b3-5e61cb4be767 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:13.352565 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f086a3-3252-499c-a52c-a46a4ee2cf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:13.353160 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f086a3-3252-499c-a52c-a46a4ee2cf74 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.353439 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f086a3-3252-499c-a52c-a46a4ee2cf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.353743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f086a3-3252-499c-a52c-a46a4ee2cf74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.354375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f086a3-3252-499c-a52c-a46a4ee2cf74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:13.356266 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f086a3-3252-499c-a52c-a46a4ee2cf74 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:13.383882 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1452/2909] 10.222.0.100 () {60 vars in 1158 bytes} [Thu Sep 1 17:48:13 2022] POST /identity/v3/projects => generated 364 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:48:13.388297 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2afd4-be23-4cce-8325-907319403857 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:13.399145 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2afd4-be23-4cce-8325-907319403857 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:13.455084 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190fa3-9e03-41e9-abc4-3cd80774392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:13.455754 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190fa3-9e03-41e9-abc4-3cd80774392d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.455948 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190fa3-9e03-41e9-abc4-3cd80774392d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.456218 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190fa3-9e03-41e9-abc4-3cd80774392d admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.456878 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd190fa3-9e03-41e9-abc4-3cd80774392d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:13.459481 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd190fa3-9e03-41e9-abc4-3cd80774392d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:13.480534 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1458/2910] 10.222.0.100 () {60 vars in 1149 bytes} [Thu Sep 1 17:48:13 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:48:13.484628 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f086a3-3252-499c-a52c-a46a4ee2cf74 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:13.495084 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f086a3-3252-499c-a52c-a46a4ee2cf74 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:13.499861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1d289-9d78-4dc8-a6f9-3921e13d0379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:13.500458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1d289-9d78-4dc8-a6f9-3921e13d0379 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.500736 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1d289-9d78-4dc8-a6f9-3921e13d0379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.501005 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1d289-9d78-4dc8-a6f9-3921e13d0379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.501719 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1d289-9d78-4dc8-a6f9-3921e13d0379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:13.503277 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1d289-9d78-4dc8-a6f9-3921e13d0379 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:13.514352 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1453/2911] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:48:13 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:48:13.518501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190fa3-9e03-41e9-abc4-3cd80774392d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:13.529221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190fa3-9e03-41e9-abc4-3cd80774392d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:13.533840 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de925ba8-1d1a-4948-a482-c7d6d97336db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:13.534464 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de925ba8-1d1a-4948-a482-c7d6d97336db admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.534739 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de925ba8-1d1a-4948-a482-c7d6d97336db admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.535008 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de925ba8-1d1a-4948-a482-c7d6d97336db admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.535664 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de925ba8-1d1a-4948-a482-c7d6d97336db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:13.537598 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de925ba8-1d1a-4948-a482-c7d6d97336db admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:13.548516 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1459/2912] 10.222.0.100 () {58 vars in 1127 bytes} [Thu Sep 1 17:48:13 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:48:13.552721 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1d289-9d78-4dc8-a6f9-3921e13d0379 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:13.562616 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1d289-9d78-4dc8-a6f9-3921e13d0379 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:13.567247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bb67a-6fc5-4740-a9e9-55a6d261154a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:13.568484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bb67a-6fc5-4740-a9e9-55a6d261154a admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.568766 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bb67a-6fc5-4740-a9e9-55a6d261154a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.569040 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bb67a-6fc5-4740-a9e9-55a6d261154a admin admin] PATH_INFO: `/v3/projects/e0698e2c3e7f42b1876e95127b82a83a/users/6e215d6b76f24a53a19a9ee451c1f72b/roles/90949fa670c2407ca92b16156e52f47f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.585195 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bb67a-6fc5-4740-a9e9-55a6d261154a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0698e2c3e7f42b1876e95127b82a83a, user_id=6e215d6b76f24a53a19a9ee451c1f72b, role_id=90949fa670c2407ca92b16156e52f47f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:13.587858 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bb67a-6fc5-4740-a9e9-55a6d261154a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:13.600861 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1454/2913] 10.222.0.100 () {60 vars in 1488 bytes} [Thu Sep 1 17:48:13 2022] PUT /identity/v3/projects/e0698e2c3e7f42b1876e95127b82a83a/users/6e215d6b76f24a53a19a9ee451c1f72b/roles/90949fa670c2407ca92b16156e52f47f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:48:13.611286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d913b-0552-45a7-b027-e24af63d187b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.611570 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d913b-0552-45a7-b027-e24af63d187b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.611874 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d913b-0552-45a7-b027-e24af63d187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.640082 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f21d913b-0552-45a7-b027-e24af63d187b None None] MFA Rules not processed for user `6e215d6b76f24a53a19a9ee451c1f72b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:13.641287 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f21d913b-0552-45a7-b027-e24af63d187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:13.707169 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1460/2914] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:48:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:48:13.719262 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bb67a-6fc5-4740-a9e9-55a6d261154a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:13.720988 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-036bb67a-6fc5-4740-a9e9-55a6d261154a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:13.743451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:13.783741 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bb67a-6fc5-4740-a9e9-55a6d261154a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:13.789505 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d56665-060c-4ad4-b216-a9b6a642fe81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d47d8e089f446d3bb4b6609d7256ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:13.790405 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d56665-060c-4ad4-b216-a9b6a642fe81 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:13.790750 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d56665-060c-4ad4-b216-a9b6a642fe81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:13.791029 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d56665-060c-4ad4-b216-a9b6a642fe81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:13.800928 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d56665-060c-4ad4-b216-a9b6a642fe81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:13.803092 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d56665-060c-4ad4-b216-a9b6a642fe81 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:13.812518 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1455/2915] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:48:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:48:23.704265 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:23.808178 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=17 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:26.854352 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cb4e2-15bf-41af-9083-81d9714e801d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:26.855010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cb4e2-15bf-41af-9083-81d9714e801d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:26.855010 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cb4e2-15bf-41af-9083-81d9714e801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:26.874806 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-628cb4e2-15bf-41af-9083-81d9714e801d None None] MFA Rules not processed for user `cf0dc05b996e4aa3a1d7605028c64f30`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:26.875999 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-628cb4e2-15bf-41af-9083-81d9714e801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:26.935881 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1461/2916] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:48:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:48:27.258207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d56665-060c-4ad4-b216-a9b6a642fe81 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:27.259723 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-14d56665-060c-4ad4-b216-a9b6a642fe81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:27.322801 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d56665-060c-4ad4-b216-a9b6a642fe81 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:27.326441 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:27.327135 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:27.327458 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:27.327795 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:27.329597 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:27.340387 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:27.341837 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:27.350465 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1456/2917] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:48:27 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:48:27.826081 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cb4e2-15bf-41af-9083-81d9714e801d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:27.827694 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-628cb4e2-15bf-41af-9083-81d9714e801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:27.908059 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cb4e2-15bf-41af-9083-81d9714e801d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:27.913547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc3ef2-37ff-4fb0-8b74-a2f41d55a502 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:27.914176 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc3ef2-37ff-4fb0-8b74-a2f41d55a502 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:27.914453 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc3ef2-37ff-4fb0-8b74-a2f41d55a502 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:27.914720 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc3ef2-37ff-4fb0-8b74-a2f41d55a502 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:27.916411 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fedc3ef2-37ff-4fb0-8b74-a2f41d55a502 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:27.928768 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc3ef2-37ff-4fb0-8b74-a2f41d55a502 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:27.930442 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc3ef2-37ff-4fb0-8b74-a2f41d55a502 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:27.940713 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1462/2918] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:48:27 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:48:37.346397 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=26 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:37.936180 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=57 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:40.437908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:40.439413 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:40.516539 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665545fc-26ca-4bc0-b5a0-be8cfa79c67b service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:40.522375 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31388c71-a5ea-49d6-b33f-ef201efc4599 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:40.523038 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31388c71-a5ea-49d6-b33f-ef201efc4599 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:40.523402 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31388c71-a5ea-49d6-b33f-ef201efc4599 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:40.523680 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31388c71-a5ea-49d6-b33f-ef201efc4599 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:40.525453 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-31388c71-a5ea-49d6-b33f-ef201efc4599 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:40.537787 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31388c71-a5ea-49d6-b33f-ef201efc4599 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:40.540002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31388c71-a5ea-49d6-b33f-ef201efc4599 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:40.551239 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1457/2919] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:48:40 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:48:50.546618 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=34 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:51.249227 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc3ef2-37ff-4fb0-8b74-a2f41d55a502 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:51.312559 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc3ef2-37ff-4fb0-8b74-a2f41d55a502 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:51.317088 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:51.317734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:51.318062 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:51.318334 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] PATH_INFO: `/v3/users/6e215d6b76f24a53a19a9ee451c1f72b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:51.319438 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e215d6b76f24a53a19a9ee451c1f72b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:51.321706 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:51.368514 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e215d6b76f24a53a19a9ee451c1f72b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.381206 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e215d6b76f24a53a19a9ee451c1f72b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.388221 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e215d6b76f24a53a19a9ee451c1f72b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.395528 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e215d6b76f24a53a19a9ee451c1f72b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.396130 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e215d6b76f24a53a19a9ee451c1f72b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.402691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1463/2920] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:48:51 2022] DELETE /identity/v3/users/6e215d6b76f24a53a19a9ee451c1f72b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:48:51.407312 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be860-95a8-4d0e-9004-f4e775ce0cbc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:51.407601 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be860-95a8-4d0e-9004-f4e775ce0cbc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:51.407906 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be860-95a8-4d0e-9004-f4e775ce0cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:51.427558 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-273be860-95a8-4d0e-9004-f4e775ce0cbc None None] MFA Rules not processed for user `d51a342066d64ae3b06abd4744cb8e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:51.428743 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-273be860-95a8-4d0e-9004-f4e775ce0cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:51.486087 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1458/2921] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:48:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:48:51.496378 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:51.497812 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:51.554937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeabc9e-bba6-40d6-b262-df5622477e67 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:51.558582 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5b459-f48a-4201-9f65-04d21cd1384e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:51.559174 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5b459-f48a-4201-9f65-04d21cd1384e service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:51.559446 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5b459-f48a-4201-9f65-04d21cd1384e service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:51.559711 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5b459-f48a-4201-9f65-04d21cd1384e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:51.568677 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5b459-f48a-4201-9f65-04d21cd1384e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:51.570657 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5b459-f48a-4201-9f65-04d21cd1384e service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:51.579951 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1464/2922] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:48:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:48:51.859908 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273be860-95a8-4d0e-9004-f4e775ce0cbc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:51.861424 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-273be860-95a8-4d0e-9004-f4e775ce0cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:51.877504 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273be860-95a8-4d0e-9004-f4e775ce0cbc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:51.884002 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:51.884638 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:51.884974 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:51.885169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] PATH_INFO: `/v3/projects/e0698e2c3e7f42b1876e95127b82a83a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:51.886347 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0698e2c3e7f42b1876e95127b82a83a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:51.888333 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:51.935995 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0698e2c3e7f42b1876e95127b82a83a', 'request_id': 'req-8a70155d-33cb-42fc-89e6-33bb848eacf8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.941939 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0698e2c3e7f42b1876e95127b82a83a', 'request_id': 'req-8a70155d-33cb-42fc-89e6-33bb848eacf8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.947522 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0698e2c3e7f42b1876e95127b82a83a', 'request_id': 'req-8a70155d-33cb-42fc-89e6-33bb848eacf8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.959470 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] The token cache is being invalidate because project e0698e2c3e7f42b1876e95127b82a83a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:48:51.959737 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.960373 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8a70155d-33cb-42fc-89e6-33bb848eacf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:48:51.962378 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1459/2923] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:48:51 2022] DELETE /identity/v3/projects/e0698e2c3e7f42b1876e95127b82a83a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:49:01.575960 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:01.956540 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:48.821521 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5b459-f48a-4201-9f65-04d21cd1384e service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:48.822956 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e0b5b459-f48a-4201-9f65-04d21cd1384e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:49:48.892967 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5b459-f48a-4201-9f65-04d21cd1384e service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:48.954748 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:48.955353 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:48.955625 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:48.955892 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:48.957646 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:49:48.970114 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:48.971670 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:48.981842 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1465/2924] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:49:48 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:49:58.978113 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=58 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:50:28.212247 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e451d-dda3-4900-968a-68eb8af75dcc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:50:28.212821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e451d-dda3-4900-968a-68eb8af75dcc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:50:28.212821 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e451d-dda3-4900-968a-68eb8af75dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:50:28.232692 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a20e451d-dda3-4900-968a-68eb8af75dcc None None] MFA Rules not processed for user `cf0dc05b996e4aa3a1d7605028c64f30`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:50:28.233942 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a20e451d-dda3-4900-968a-68eb8af75dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:28.288490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1460/2925] 10.222.0.100 () {58 vars in 963 bytes} [Thu Sep 1 17:50:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:50:38.281935 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=25 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:50:40.105417 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:50:40.107286 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:40.192937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4258488-6092-440b-86a9-2cd30ff6bfbb service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:50:40.198644 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:50:40.199266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:50:40.199553 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:50:40.199826 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:50:40.201691 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:40.213743 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:50:40.215314 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:50:40.224820 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1466/2926] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:50:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:50:47.154517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e451d-dda3-4900-968a-68eb8af75dcc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:50:47.155992 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a20e451d-dda3-4900-968a-68eb8af75dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:47.173073 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e451d-dda3-4900-968a-68eb8af75dcc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:50:47.179850 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45499fc5-c59f-4730-9b6a-83377142ae93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:50:47.180818 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45499fc5-c59f-4730-9b6a-83377142ae93 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:50:47.181233 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45499fc5-c59f-4730-9b6a-83377142ae93 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:50:47.181517 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45499fc5-c59f-4730-9b6a-83377142ae93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:50:47.183553 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-45499fc5-c59f-4730-9b6a-83377142ae93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:50:47.198139 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45499fc5-c59f-4730-9b6a-83377142ae93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:50:47.200000 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45499fc5-c59f-4730-9b6a-83377142ae93 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:50:47.215218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1461/2927] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:50:47 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:50:50.220450 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=34 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:50:57.208547 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:15.679793 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:15.681432 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:15.744501 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170af4c3-5b45-4b77-9ed9-5eeb2ae82d5a service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:15.748137 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae28f60-3618-4650-8b59-73902672c2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:15.748753 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae28f60-3618-4650-8b59-73902672c2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:15.749031 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae28f60-3618-4650-8b59-73902672c2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:15.749313 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae28f60-3618-4650-8b59-73902672c2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:15.751165 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cae28f60-3618-4650-8b59-73902672c2e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:15.766911 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae28f60-3618-4650-8b59-73902672c2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:15.769266 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae28f60-3618-4650-8b59-73902672c2e6 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:15.781757 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1467/2928] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:15 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:16.962524 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45499fc5-c59f-4730-9b6a-83377142ae93 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:16.972550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45499fc5-c59f-4730-9b6a-83377142ae93 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:16.976211 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b201c5-2bca-472b-83d7-c092681bd3be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:16.976809 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b201c5-2bca-472b-83d7-c092681bd3be service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:16.977090 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b201c5-2bca-472b-83d7-c092681bd3be service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:16.977361 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b201c5-2bca-472b-83d7-c092681bd3be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:16.979093 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c5b201c5-2bca-472b-83d7-c092681bd3be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:16.993174 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b201c5-2bca-472b-83d7-c092681bd3be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:16.994615 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b201c5-2bca-472b-83d7-c092681bd3be service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:17.005704 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1462/2929] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:16 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:25.076488 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae28f60-3618-4650-8b59-73902672c2e6 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:25.090890 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae28f60-3618-4650-8b59-73902672c2e6 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:25.096833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34f6f3e93b54a5d92b6f2f9f0de40f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:25.097563 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:25.098520 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:25.098867 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:25.100833 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:25.114034 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:25.115943 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:25.128468 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1468/2930] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:25 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:27.000677 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:28.363201 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b201c5-2bca-472b-83d7-c092681bd3be service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:28.372999 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b201c5-2bca-472b-83d7-c092681bd3be service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:28.377049 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbe180b45f2e49e2abbf6e833ce0d44c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:28.377701 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:28.377979 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:28.378241 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:28.381098 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:28.446207 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:28.447653 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:28.458816 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1463/2931] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:31.687734 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:31.693947 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:31.722207 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557abe8f-a7df-45d1-8681-106b1889daae service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:31.730623 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2330b-914c-42a7-aeed-db52757fe758 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:31.731346 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2330b-914c-42a7-aeed-db52757fe758 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:31.732281 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2330b-914c-42a7-aeed-db52757fe758 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:31.732629 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2330b-914c-42a7-aeed-db52757fe758 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:31.734493 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-12b2330b-914c-42a7-aeed-db52757fe758 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:31.748299 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2330b-914c-42a7-aeed-db52757fe758 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:31.750138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2330b-914c-42a7-aeed-db52757fe758 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:31.763764 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1469/2932] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:52:31 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:52:35.343764 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:35.346445 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:35.359842 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa06906-fed6-4d00-969a-d3e1d2dac4d8 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:35.367432 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:35.368218 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:35.368502 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:35.368778 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] PATH_INFO: `/v3/users/cf0dc05b996e4aa3a1d7605028c64f30` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:35.370147 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0dc05b996e4aa3a1d7605028c64f30)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:35.372169 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:35.429163 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0dc05b996e4aa3a1d7605028c64f30'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.438550 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0dc05b996e4aa3a1d7605028c64f30'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.439559 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0dc05b996e4aa3a1d7605028c64f30'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.454989 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0dc05b996e4aa3a1d7605028c64f30'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.463676 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0dc05b996e4aa3a1d7605028c64f30'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.470900 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1464/2933] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:52:35 2022] DELETE /identity/v3/users/cf0dc05b996e4aa3a1d7605028c64f30 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:52:35.717371 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2330b-914c-42a7-aeed-db52757fe758 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:35.732550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2330b-914c-42a7-aeed-db52757fe758 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:35.811097 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:35.812138 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:35.812513 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:35.812916 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] PATH_INFO: `/v3/projects/2e0189e8a6ef4f2c930f0864100a9d82` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:35.814325 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0189e8a6ef4f2c930f0864100a9d82)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:35.817813 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:35.876300 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0189e8a6ef4f2c930f0864100a9d82', 'request_id': 'req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.883800 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0189e8a6ef4f2c930f0864100a9d82', 'request_id': 'req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.890896 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0189e8a6ef4f2c930f0864100a9d82', 'request_id': 'req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.903156 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] The token cache is being invalidate because project 2e0189e8a6ef4f2c930f0864100a9d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:52:35.903439 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.904104 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:35.905698 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1470/2934] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:52:35 2022] DELETE /identity/v3/projects/2e0189e8a6ef4f2c930f0864100a9d82 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:35.912016 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:35.914047 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:35.929451 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bbd9c-d910-40f3-a66c-d5ed4f92e906 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:36.000902 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:36.001444 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:36.001819 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:36.002069 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] PATH_INFO: `/v3/users/8bdb8c5bcbc94fea93efb84814a5ffca` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:36.003295 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bdb8c5bcbc94fea93efb84814a5ffca)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:36.005805 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:36.012287 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:36.064485 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bdb8c5bcbc94fea93efb84814a5ffca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.073892 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bdb8c5bcbc94fea93efb84814a5ffca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.074534 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bdb8c5bcbc94fea93efb84814a5ffca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.088212 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bdb8c5bcbc94fea93efb84814a5ffca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.095484 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bdb8c5bcbc94fea93efb84814a5ffca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.102490 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1465/2935] 10.222.0.100 () {58 vars in 1229 bytes} [Thu Sep 1 17:52:35 2022] DELETE /identity/v3/users/8bdb8c5bcbc94fea93efb84814a5ffca => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:36.306937 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:52:36.308733 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:52:36.324972 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402ad00-bb48-4cb7-b1e0-1dafa0d6ef4d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:52:36.331048 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:36.398144 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51a342066d64ae3b06abd4744cb8e8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392e00333a6b41639a27c8ea706c3db5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:52:36.399154 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:52:36.399587 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:52:36.399894 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] PATH_INFO: `/v3/projects/80e13821282c4a129d1c943b531e6721` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:52:36.401394 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e13821282c4a129d1c943b531e6721)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:52:36.403362 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:52:36.459240 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e13821282c4a129d1c943b531e6721', 'request_id': 'req-373bcb3d-5a34-4aa8-b608-afed61afcb3c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.467692 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e13821282c4a129d1c943b531e6721', 'request_id': 'req-373bcb3d-5a34-4aa8-b608-afed61afcb3c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.475547 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e13821282c4a129d1c943b531e6721', 'request_id': 'req-373bcb3d-5a34-4aa8-b608-afed61afcb3c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.496880 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] The token cache is being invalidate because project 80e13821282c4a129d1c943b531e6721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 17:52:36.497398 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.498550 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-373bcb3d-5a34-4aa8-b608-afed61afcb3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 17:52:36.501120 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1471/2936] 10.222.0.100 () {58 vars in 1238 bytes} [Thu Sep 1 17:52:36 2022] DELETE /identity/v3/projects/80e13821282c4a129d1c943b531e6721 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:52:46.099567 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:52:46.490830 dv-cinder-839768-6-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:54:54.768301 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:54:54.769756 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:54:54.924200 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e765877-63b8-43a7-8cf5-61861cb3cfe9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:54:54.986804 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606591d-3220-4438-a916-7894964ccaff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b181d38588d44d4bb72fa319d2ba2f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '134fe94171684b8d96893638e4870bcc', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:54:54.987488 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606591d-3220-4438-a916-7894964ccaff service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:54:54.987815 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606591d-3220-4438-a916-7894964ccaff service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:54:54.988094 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606591d-3220-4438-a916-7894964ccaff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:54:54.990720 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1606591d-3220-4438-a916-7894964ccaff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:54:55.007905 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606591d-3220-4438-a916-7894964ccaff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:54:55.009797 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606591d-3220-4438-a916-7894964ccaff service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:54:55.033449 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1466/2937] 10.222.0.100 () {60 vars in 1370 bytes} [Thu Sep 1 17:54:54 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:55:05.023499 dv-cinder-839768-6-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}